chickadee » nanograd » cross-entropy-loss

Cross-entropy loss with batch support.

pred

predictions: probabilities (softmax already applied) when from-logits is #f, or raw scores when it is #t

target

one-hot targets of the same shape as pred, or class indices of shape (batch_size)

reduction

'mean (average over the batch) or 'sum

from-logits

if true, apply log-softmax to pred first

Input shapes:

• 1-D pred (n_classes): single sample
• 2-D pred (batch_size, n_classes): batch of samples

; Single sample with one-hot target
(define loss (cross-entropy-loss probs target))

; Batch with one-hot targets
(define batch-probs (softmax logits axis: -1))
(define batch-loss (cross-entropy-loss batch-probs targets reduction: 'mean))

; Batch with class indices
(define class-indices (make-tensor32 (f32vector 0.0 2.0 1.0) '(3)))
(define batch-loss (cross-entropy-loss logits class-indices
                                       from-logits: #t
                                       reduction: 'mean))