chickadee » nanograd » log-softmax

log-softmax x #!key (axis -1)procedure

Log-softmax with batch support. It is more numerically stable than log(softmax(x)).

Input shapes:

  • 1-D: (n_classes)
  • 2-D: (batch_size, n_classes) - log-softmax along axis

Gradient: dL/dx = dL/dy - exp(log_softmax(x)) * sum(dL/dy)