- register-optimizer-kernel! name dtype kernelprocedure
Uses kernel for the updates of optimizer name (a symbol, such as 'adam) on parameters of dtype 'f32 or 'f64. A kernel of #f removes the registration.
An Adam kernel is a procedure (kernel n hyper param grad m v) that updates the first n elements of the SRFI-4 vectors param, m and v in place, and adds the weight-decay term to grad. hyper is an f64vector holding the learning rate, beta1, beta2, epsilon, the bias corrections 1 - beta1^t and 1 - beta2^t, and the weight decay.