- check-gradient-health parameters #!key (exploding-threshold 10.0) (vanishing-threshold 1e-07) (imbalance-threshold 1000.0) (warn? #t)procedure
Checks for exploding, vanishing and unbalanced gradients. Gradients are unbalanced when the largest per-parameter L2 norm is more than imbalance-threshold times the smallest one; this check is skipped when some gradient is zero. With warn?, each problem found is printed.
Returns an association list with healthy? (#t if no problem was found), issues (a list of symbols, such as gradient-imbalance) and stats (the result of gradient-stats).