chickadee » zlib

chicken-zlib

Bindings to the ubiquitous zlib library. zlib can compress and decompress:

The default compression output for zlib and this egg is RFC1950. Use the #:window-bits keyword argument to change this.

As of version 0.8, this egg repository replaces the previous zlib egg. It is a rewrite but aims to be a drop-in replacement. See below for details.

Requirements

Source code

The repository is hosted here.

API

zlib-compressing-output-port port #!rest optionsprocedure

Returns an output-port to which arbitrary data can be written. Its compressed form will be written to, usually with some delay, the supplied output-port port. It is important to call close-output-port on the returned port. Doing so does not close the supplied port.

The keyword arguments options are supplied to deflateInit2. They are as follows:

  • level: compression level in the range [0-9] (from fastest to best). 0 means no compression is applied and the data is simply wrapped in the associated headers. The current default is level 6.
  • method: Currently only 'deflated is supported.
  • window-bits: Specifies the history buffer size in base two logarithm. The current default is 15. Supported ranges are:
    • [8..15] for zlib RFC1950 streams.
    • [-15..-8] for raw deflate RFC1951 streams.
    • [25..31] for gzip RFC1952 streams.
  • mem-level: Specifies memory consumption for internal state, more is faster. Valid ranges are [1..9]. The current default is 8.
  • strategy: Valid symbols are
    • 'default (or #f)
    • 'filtered
    • 'huffman-only
    • 'rle
    • 'fixed
  • set-finalizer: A procedure called on the resulting output-port. The default is (lambda (x) (set-finalizer! x deflate-free!)). Supply your own if set-finalizer!'s overhead is undesirable.
  • buffer: A string, often heavily mutated, used for internal transfers. (make-string 4096) is the default.

If some of the supplied options are invalid, zlib throws (error ...). Note that flush-output-port currently has no affect. Although zlib has support to flush the stream, providing immediate available data, this degrades the compression performance. As this is usually not the desired outcome of flush-output-port, we leave it as a no-op.

Here is an example to produce zlib compressed data:

(string->blob
 (call-with-output-string
  (lambda (os)
    (let ((op (zlib-compressing-output-port os)))
      (display "hello world" op)
      (close-output-port op)))))
;; => #${789ccb48cdc9c95728cf2fca4901001a0b045d}
;; echo  789ccb48cdc9c95728cf2fca4901001a0b045d | xxd -plain -revert | file -
/dev/stdin: zlib compressed data
zlib-decompressing-input-port ip #!key window-bits bufferprocedure

These options are passed to inflateInit2.

  • window-bits: The history buffer size. When decompressing a stream, the window size must not be smaller than the size originally used to compress the stream. Valid ranges are:
    • 0 to automatically detect the window size from the zlib header (supported since zlib 1.2.3.5)
    • [8..15] for zlib RFC1950 streams.
    • [−8..−15] for raw deflate RFC1951 streams.
    • [24..31] for gzip RFC1952 streams.
    • [40..47] for either zlib or gzip streams, automatically detected by header.
  • buffer: A string, often heavily mutated, used for internal transfers. (make-string 4096) is the default.

For example, we can decompress the zlib data from the example above like this:

(read-string #f
  (zlib-decompressing-input-port
     (open-input-string
      (blob->string #${789ccb48cdc9c95728cf2fca4901001a0b045d}))))
;; => "hello world"

This will expect zlib headers. To detect gzip or zlib headers, specify higher values for window-bits:.

;; $ printf "hello world" | pigz -z | xxd -plain -c0
;; 785ecb48cdc9c95728cf2fca4901001a0b045d
(define hello.zlib (blob->string #${785ecb48cdc9c95728cf2fca4901001a0b045d}))
;; $ printf "hello world" | gzip - | xxd -plain -c0
;; 1f8b0800000000000003cb48cdc9c95728cf2fca49010085114a0d0b000000
(define hello.gz (blob->string #${1f8b0800000000000003cb48cdc9c95728cf2fca49010085114a0d0b000000}))
(read-string #f (zlib-decompressing-input-port (open-input-string hello.zlib) #:window-bits 47))
;; => "hello world"
(read-string #f (zlib-decompressing-input-port (open-input-string hello.gz)   #:window-bits 47))
;; => "hello world"

Examples

For more examples, see the ./examples directory. Example usage:

$ echo hello world | gzip -9 | csi -s examples/unzlib.scm
hello world
$ echo hello world | csi -s examples/zlib.scm | file -
/dev/stdin: zlib compressed data
$ pv -Ss8G /dev/zero | gzip | csi -s examples/unzlib.scm >/dev/null
8.00GiB 0:00:12 [ 680MiB/s] [====================================>] 100%

History

This is a replacement for r1b's zlib egg, and the new repository is here. The maintainer role has been transferred. The two eggs do not share any code, but version 0.8 onwards shares a lot of code with chicken-zstd.

It provides the same two procedures as 0.7, but these have been deprecated:

Here is an outline of a few other changes:

TODOs

string-string API

Some compression algorithm libraries provide procedures to compress and decompress strings directly, without using ports. These may be more convenient, but have size limits. Because of this, and since Scheme ports are relatively easy to use, only this port-based API is provided. The author is open for suggestions.

Contents »