chickadee » lexgen » tok

tok <Input>procedure

Procedure tok builds pattern matchers based on character comparison operations. It is intended for matching input sequences of arbitrary kinds, e.g. character lists, strings, or other kinds of sequences. To achieve abstraction over the input sequence kind, tok is parameterised on a type class named <Input>. Please see libraries typeclass and input-classes for information on the type class interface.

As an example, the code below creates an input class for character lists and defines a version of tok specialized for character lists.

(require-extension typeclass input-classes)

(define char-list-<Input>
  (make-<Input> null? car cdr))

(define char-list-tok (tok <char-list-<Input>))

Once applied to an input class, tok builds a pattern matcher that, for each stream given, applies a procedure to the given token TOKEN and an input character. If the procedure returns a true value, that value is prepended to the list of consumed elements, and the input character is removed from the list of input elements.