chickadee » pserializer

pserializer

A portable serializer for Scheme.

The serializer converts Scheme objects to a byte stream that can be stored in a file or sent over a network, and reads the stream back into an equivalent object structure. Shared and circular structure is preserved: objects that appear more than once keep their eq? identity after a round trip.

The core module handles standard Scheme data. SRFI-4 homogeneous numeric vectors, SRFI-69 hash tables, keywords, and blobs are supported through a registration facility, which applications can also use to add handlers for their own types. An optional companion module compresses serialized output with the zlib stream format.

This is a CHICKEN 6 adaptation of the portable serializer from Shiro Kawai's STk / Gauche serializer package. The wire format is unchanged, so streams written by the original implementation can still be read.

Documentation

The egg installs two extensions:

pserializer
The serializer core.
pserializer-deflate
Compression support with a bundled copy of miniz. Importing this module enables the compress keyword of the core procedures. The core module does not import it, so the core can be used without compression.

Serializable types

The core handles these types:

Registered extensions add:

Procedures, records created by define-record, ports, and other implementation-specific objects are not serializable unless an extension is registered for them.

Shared and circular structure round-trips with identity preserved. Two occurrences of the same string in a tree are again the same string after a round trip; a cyclic list or vector reads back as the same cycle.

Serialized format

The format is text and uses the Scheme external representation throughout.

|<th>Tag</th><th>Object</th><th>Payload</th> |y | symbol | the symbol, in external representation |p | pair | serialized car, then serialized cdr |s | string | the string, in external representation |v | vector | element count, then each element |h | hash table | entry count, then key/value pairs |k | keyword | keyword name as a symbol |b | blob | byte count, then raw bytes |c | compressed | byte count, then raw bytes of a zlib stream |r | backref | reference number of a previously written object

Binary payloads (blob contents, homogeneous vector data, compressed data) are embedded as raw bytes between lines; byte counts are always written as text.

SRFI-4 vectors carry a type name, an element count, and the raw elements. Integer elements are stored little-endian, so the format is independent of host byte order. Floating point elements are stored as IEEE 754 bit patterns, also little-endian.

A compressed frame holds a complete serialized object. The reader decompresses the frame and parses its contents from memory, so a stream may mix compressed and uncompressed frames and still share back-references across frame boundaries.

Flonums are written with 18 significant digits so that every double precision value round-trips bit-exactly.

Module: pserializer

(serializer-write OBJECT PORT [#:compress FLAG])procedure

Writes one object to PORT as a serialized stream. With #:compress #t the object is written as one compressed frame; this requires the pserializer-deflate extension to be loaded.

serializer-read PORTprocedure

Reads one object from PORT. Returns an eof object when the stream holds no more objects. Compressed frames are decompressed transparently.

(serializer->string OBJECT [#:compress FLAG])procedure

Serializes one object to a fresh string.

string->serializer STRINGprocedure

Deserializes one object from STRING. The string must hold exactly one serialized object.

(make-output-serializer PORT [#:extensions LIST] [#:compress FLAG])procedure

Creates an output serializer that writes to PORT. LIST is a list of extension specifications (see below). With #:compress #t every object written through this serializer is emitted as a compressed frame.

write-to-output-serializer OBJECT SERIALIZERprocedure

Writes one object to an output serializer.

(call-with-output-serializer PORT PROC [#:extensions LIST] [#:compress FLAG])procedure

Calls PROC with one argument, an output serializer on PORT. Returns the value of PROC.

(make-input-serializer PORT [#:extensions LIST])procedure

Creates an input serializer that reads from PORT.

read-from-input-serializer SERIALIZERprocedure

Reads one object from an input serializer. Returns an eof object at the end of the stream.

(call-with-input-serializer PORT PROC [#:extensions LIST])procedure

Calls PROC with one argument, an input serializer on PORT. Returns the value of PROC.

<accessor>(serializer->port SERIALIZER)</accessor>

Returns the port associated with a serializer.

register-serializer-extension! TAG TEST WRITER READERprocedure

Registers a handler for one additional object type, globally.

TAG
a symbol naming the type in the serialized stream.
TEST
a predicate; it returns true for objects the extension can serialize.
WRITER
called with two arguments, the object and the output serializer, and must emit the object's payload. The tag itself is written by the serializer before WRITER runs. Payload elements that are ordinary objects should be written with write-to-output-serializer, so that sharing is preserved.
READER
called with one argument, the input serializer, and must read the payload and return the reconstructed object. Payload elements should be read with read-from-input-serializer.

Later registrations take priority over earlier ones, and per-serializer extensions (the extensions: keyword of the constructors) take priority over registered ones.

serializer-extensionsparameter

Holds the list of globally registered extension specifications. It is initialized with the built-in extensions for SRFI-4 vectors, hash tables, keywords, blobs, and compressed frames.

register-object-to-input-serializer OBJECT SERIALIZER #!optional COUNTprocedure

Registers OBJECT under reference number COUNT, or under the next reference number when COUNT is omitted. Extension readers that build an object whose payload contains back-references to itself must call this before reading those references.

pserializer-compressionparameter

When set to a true value, serializer-write and serializer->string compress their output by default. An explicit #:compress keyword argument takes priority over this parameter.

pserializer-deflate-compressparameter
pserializer-deflate-decompressparameter

Hooks that the pserializer-deflate module installs at load time. The compress hook takes a blob and its byte count and returns two values, a bytevector and the number of bytes of the zlib stream in it. The decompress hook takes a blob and its byte count and returns the decompressed payload as a blob. These hooks are exported so that other compression backends can be substituted.

Module: pserializer-deflate

Loads and bundles miniz, and installs the compression hooks described above. Importing this module is the only step needed to enable compression; the core module never imports it.

flate-compress STRINGprocedure

Compresses a byte string (characters with codes 0 to 255) to an RFC 1950 (zlib) stream, returned as a byte string.

flate-decompress STRINGprocedure

Decompresses an RFC 1950 stream to the original byte string. Signals an error for corrupt input.

flate-compress-bytevector/bv BYTEVECTOR COUNTprocedure

Compresses the first COUNT bytes of BYTEVECTOR, returning two values: a bytevector holding the zlib stream, and its length. The returned bytevector's capacity may exceed the length.

flate-compress-bytevector STRINGprocedure

Compresses a byte string, returning two values in the same style as flate-compress-bytevector/bv.

flate-decompress-bytevector BYTEVECTOR COUNTprocedure

Decompresses the first COUNT bytes of BYTEVECTOR, returning a bytevector sized exactly to the decompressed contents.

string->byte-bytevector STRINGprocedure

Converts a byte string to a bytevector, one byte per character. Signals an error for characters outside 0 to 255.

bytevector->byte-string BYTEVECTOR COUNTprocedure

Converts the first COUNT bytes of a bytevector to a byte string.

Examples

Serializing to a string and back:

(import pserializer)

(define data (list 1 2.5 "three" 'sym (vector 'a 'b)))
(define copy (string->serializer (serializer->string data)))

Writing several objects to one stream, with compression:

(import pserializer pserializer-deflate)

(define p (open-output-string))
(serializer-write '(1 2 3) p)
(serializer-write '(4 5 6) p compress: #t)
(serializer-write '(7 8 9) p)

(define in (open-input-string (get-output-string p)))
(serializer-read in)   ; => (1 2 3)
(serializer-read in)   ; => (4 5 6)
(serializer-read in)   ; => (7 8 9)
(serializer-read in)   ; => an eof object

Registering an extension for a record type:

(import pserializer)

(define-record point x y)

(register-serializer-extension!
 'point                                ; tag symbol
 point?                                ; test
 (lambda (obj ser)                     ; writer: emit payload
   (write-to-output-serializer (point-x obj) ser)
   (write-to-output-serializer (point-y obj) ser))
 (lambda (ser)                         ; reader: read payload, build object
   (let ((pt (make-point 0 0)))
     ;; register before reading so back-references resolve
     (register-object-to-input-serializer pt ser)
     (let ((x (read-from-input-serializer ser))
           (y (read-from-input-serializer ser)))
       (point-x-set! pt x)
       (point-y-set! pt y)
       pt))))

(define pt2 (string->serializer (serializer->string (make-point 3 4))))

Notes on behavior

eq?-ness
Objects handled through the reference table (pairs, strings, symbols, vectors, SRFI-4 vectors, hash tables, blobs, keywords) keep their identity across a round trip. Numbers and characters bypass the table, so only their eqv?-ness is preserved.
Hash tables
The deserialized table uses eq? hashing. The original table's test and hash functions are not serialized.
Flonums
Written with 18 significant digits and read back bit-exactly.
Characters and strings
Any character code, including codes above 127 and NUL, round-trips exactly.
Errors
The writer signals an error for objects no extension accepts. The reader signals errors for unknown tags, unknown reference numbers, and truncated input.

Author

Shiro Kawai (original STk / Gauche serializer); CHICKEN adaptation by Ivan Raikov.

Repository

Source repository: pserializer.

Requirements

srfi-69

Version History

1.0
Initial release.

License

BSD 3-Clause.

 Based on the portable serializer by Shiro Kawai (1999).

 Copyright (c) 2026, Ivan Raikov
 All rights reserved.

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are
 met:

 Redistributions of source code must retain the above copyright
 notice, this list of conditions and the following disclaimer.

 Redistributions in binary form must reproduce the above copyright
 notice, this list of conditions and the following disclaimer in the
 documentation and/or other materials provided with the distribution.

 Neither the name of the copyright holder nor the names of its
 contributors may be used to endorse or promote products derived from
 this software without specific prior written permission.

 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
 FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
 COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
 BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
 OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
 TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
 USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
 DAMAGE.

Contents »