chickadee » nanograd » log-softmax

log-softmax x #!key (axis -1)procedure

Log-softmax with batch support: more numerically stable than log(softmax(x)).

Input shapes:

  • 1D: (n_classes,)
  • 2D: (batch_size, n_classes) - log-softmax along axis

Gradient: dL/dx = dL/dy - exp(log_softmax(x)) · Σ(dL/dy)