- make-sgd parameters #!key (learning-rate 0.01) (momentum 0.0) (weight-decay 0.0) (nesterov #f)procedure
Stochastic Gradient Descent optimizer with optional momentum and Nesterov acceleration.
- parameters
- list of parameter tensors to optimize
- learning-rate
- step size (default 0.01)
- momentum
- momentum factor (default 0.0, no momentum)
- weight-decay
- L2 regularization factor (default 0.0)
- nesterov
- use Nesterov momentum (default #f)