chickadee » nanograd-components » make-am-transformer

make-am-transformer vocab-size seq-len d-model n-layers n-heads #!key (d-ff #f) (use-rope? #t) (rope-base 10000.0) (causal? #t) (norm-eps 1e-05) (dtype 'f32) (name AM-Transformer)procedure

Creates the lazy model. Unlike max-seq-len in the strict model, seq-len is the fixed sequence length T.