chickadee » nanograd » make-batch-norm-2d

Creates a 2-D batch normalization layer:

y = gamma * (x - mu) / sqrt(sigma^2 + epsilon) + beta

where mu and sigma^2 are the per-channel mean and variance of the batch in training mode, and running averages of them in evaluation mode.

num-features

number of channels (C)

epsilon

small constant for numerical stability (default 1e-5)

momentum

weight of the current batch in the running statistics (default 0.1)

dtype

'f32 or 'f64 (default 'f32)

name

layer name

Input shapes:

• 3-D: (C, H, W) - treated as a batch of 1
• 4-D: (N, C, H, W) - standard batch

The output has the same shape as the input. gamma and beta are learned.

;; Batch norm for 64 channels
(define bn (make-batch-norm-2d 64 epsilon: 1e-5 momentum: 0.1))

;; Training mode: uses batch statistics
(set-training-mode! bn #t)
(define normalized (forward bn input))  ; Input shape: (N, 64, H, W)

;; Evaluation mode: uses running statistics
(set-eval-mode! bn)
(define test-normalized (forward bn test-input))