chickadee » array-morphisms » morph-lift

morph-lift fn cell-ranks arg ...procedure

Apply fn, a function on cells, to every cell of a frame as one computation. cell-ranks gives the rank of the cells that fn expects, one per argument or one for all. The leading axes of an argument, beyond its cell rank, are its frame. The longest frame becomes the result's frame, and every other frame must be a prefix of it. An argument with a shorter frame is reused along the frame axes it lacks.

;; x [F B D]: each cell is a [B D] matrix; W and b are shared by every cell
(define y (morph-lift (lambda (xc) (morph-tanh-am (morph+ (morph-matmul xc W) b)))
                      2 x))   ; cell rank 2 -> y [F B D]

morph-lift calls fn once, on placeholder arrays with the cell shapes, and widens the expression fn builds by the frame:

  • element-wise operations stay one element-wise node;
  • a reduction becomes one reduction with a frame rank;
  • a matrix product becomes one batched matrix product;
  • transposes, reshapes, slices, padding, stacks, concatenations and scans keep the frame axes in front;
  • attention takes the frame as leading axes (up to rank 4 in all), and im2col and col2im fold it into the image axis.

Parts of the expression that use no argument, such as W above, are computed once, so the expression has as many nodes for a frame of 4 cells as for 64. Every cell of the result has exactly the values that fn gives for that cell on its own.

fn must only build expressions from the placeholders; it must not read their values.