- make-cosine-schedule max-lr min-lr total-steps #!key (warmup-steps 0)procedure
Cosine annealing from max-lr down to min-lr over total-steps, after an optional linear warmup:
lr = min-lr + 0.5 * (max-lr - min-lr) * (1 + cos(pi * progress)), where progress = (step - warmup-steps) / (total-steps - warmup-steps)