chickadee » generalized-arrays

Generalized Arrays

This repository holds an implementation of an R7RS library implementing generalized arrays. As a reference, this repository is meant to build a module for CHICKEN Scheme.

Usage

Below are the instructions for using and working with the generalized-arrays egg.

Installation

 
$ chicken-install -s generalized-arrays

Importing the Library

To import the complete library, use the following import specification:

 
(import generalized-arrays)

Module Structure

The library is broken apart into several smaller modules, however. These can be individually imported as follows:

 
(import (generalized-arrays intervals)
        (generalized-arrays storage)
        (generalized-arrays arrays))

Each sub-module containing the procedures and record definitions as specified in each independent module documentation below.

Dependencies

This egg aims to be fairly light on dependencies where possible, and tries its best to be R7RS compatible. To that end, it uses the following dependencies:

• r7rs
• srfi-48
• srfi-128
• srfi-133
• srfi-160
• check-errors
• transducers (GitLab)

Most of these dependencies are SRFIs. check-errors could probably be replaced with a thin shim for type-checking arguments, and is a very small dependency. transducers is another R7RS library that is maintained and necessary for working with arrays.

Reporting Issues

Please report issues in the repository or on the CHICKEN bug tracker.

Repository

The repository can be located on GitLab.

Why Arrays?

Scheme provides a variety of contiguous data structures (e.g. vectors, SRFI-4 vectors) that can reference data over a singular index in an interval. Scheme does not, however, provide any means by default for working with data indexed over a multi-dimensional interval. In C, for example, we might declare a multi-dimensional array as follows:

 
// Declares an array `arr` that is N x N in size, holding `double`-typed values.
double (*arr)[N] = malloc(sizeof(double[N][N]));

for (size_t col = 0; col < N; ++col) {
    for (size_t row = 0; row < N; ++row) {
        if (row == col) {
            arr[col][row] = 1.0;
        }
    }
}

This can be useful for a variety of representations:

• Images holding individual pixel data at each location
• Point clouds holding multiple points with multi-dimensional data (XYZ)
• A grid of "cells" holding more structured data
• Mathematical data such as matrices, tensors, etc. depending on the rank of the array

Usually arrays are most useful at reducing complex lookup mechanisms over multiple keys into a lookup mechanism that merely uses integers within a range. This can aid in making lookup faster, but more importantly provides a means to abstract how data is "represented" in a spatial sense, even if the actual underlying representation is different.

Scheme Arrays

Unlike C (mentioned above), Scheme is dynamically typed, which opens arrays up to be much more flexibly used than their counterpart in C. However we still want to keep open the possibility of having strong(ish)ly typed arrays as well — many mathematical applications perform some construction of an array into contiguous memory expecting some layout, and then pass along said array to a much more optimized library with a specific layout to handle the larger number crunching.

The generalized-arrays egg supports arrays holding nothing but Scheme types, but also supports arrays that can be more strongly typed (e.g. by using SRFI-4 containers for array storage). This is done by separating the following concepts:

• Intervals: the shape, size, stride, etc. of an array. This describes how to traverse an array and what the underlying "shape" or structure of the array is.
• Storage: the container or record type that is used to store the array's data. This is often a "flattened" vector or vector-like container of some kind.
• Arrays: the actual objects we want to work on and their associated high-level API.

Each of these concepts is described in brief in the following sections.

Intervals

Intervals are an abstraction provided by this library to express multi-dimensional intervals expressed through integer ranges. Intervals here follow the mathematical definition of an interval, specifically, an interval can be expressed as a starting index and an ending index. In math this might look like [0 10] to express the range of integers starting at zero and up to ten, inclusive.

This library provides multi-dimensional intervals as a first class concept. One should think about intervals as being the shape or the boundaries of an array. For example, if we use the prior example of an N by N matrix, we might say that our interval is something like [(0, 0) (N - 1, N - 1)]. If this same matrix were transformed or sliced to be N by M, then we might say that the interval is [(0, 0) (N - 1, M - 1)].

Another point to note is that for the sake of API convenience this library always treats all intervals as half-open; that is, intervals are specified up-to-but-not-including their upper bound. So in the previous example where we used the inclusive "mathematical" representation of ranges, in this library you might reference a library as being [(0, 0) (N, N)) with unbalanced parentheses.

The reasons for such will become more obvious later, but if you were to create an example program to express a range in this library it might look like:

 
(import (generalized-arrays intervals))

(define N 10)

(make-interval #(0 0) #(N N))

This interval is half-open, in that it does not include the index #(N N). Indices in the generalized-arrays egg are expressed as Scheme vectors containing fixnums describing each dimension. An array can be any number of dimensions, but when making an interval the number of dimensions (the rank) must match.

NOTE: The generalized-arrays egg always assumes that all indices (i.e. the members of an interval) are represented as Scheme vectors holding non-negative fixnums.

For more information on intervals, see the API reference on intervals below.

Storage

The next core concept in this library is storage. generalized-arrays provides what are known as "storage classes" which comprise APIs that relate to how data in an array are stored. This part of the API generally does not require as much familiarity as default storage classes are provided:

• vector-storage-class
• u8vector-storage-class
• s8vector-storage-class
• u16vector-storage-class
• s16vector-storage-class
• u32vector-storage-class
• s32vector-storage-class
• u64vector-storage-class
• s64vector-storage-class
• f32vector-storage-class
• f64vector-storage-class
• c64vector-storage-class
• c128vector-storage-class

Each of these "storage classes" are designed to be instances of a unique record type that holds all the procedures necessary for working with these storage classes. This pattern is sometimes known as the type-class pattern, and you may be familiar with it if you have ever used SRFI-128 (Comparators). If not, fret not, because in all likelihood you won't have to touch most of this API unless you are making your own storage class.

In any case, the storage class chosen defines how the array will store the data. Data is stored in continguous, one-dimensional storage (e.g. it is not a set of nested vectors, or vectors of f32 vectors, or lists of vectors, or whatever have you). Most usage of storage classes will pertain to getting the storage object (which is an object that a storage class can operate on), or specifying which storage class one wishes to use as the "backing storage" of an array.

For more information on storage and storage classes, see the API reference on intervals below.

Arrays

The arrays API may appear a bit sparse initially, but this library provides every procedure necessary in order to be able to perform any individual array operation. This includes but is not limited to:

• Array traversal
• Array slicing
• Copying
• Getting array metadata (shape, rank, stride, etc)
• Checking equality
• Comparing arrays
• Setting and getting values from the array
• Printing an array to the REPL
• etc.

Many procedures which you may be accustomed to in a data-structure library may appear missing; this is because this library is meant to be used exclusively with the transducers library. It is advised to become acquainted with transducers before trying to dig too deep into what is missing from the arrays API.

Some History

This project started many years ago when I was in grad school, looking for something analagous to Numpy in Scheme. At the time, I made an attempt to implement the API that eventually became SRFIs 122, 179, and 231. To my credit, early versions of this library predated those SRFIs by quite some time; conversely, to my detriment my early implementation was pretty poor.

I did not like what eventually became SRFI 231 (supercedes all the other aforementioned SRFIs) for a handful of reasons:

1. The API is largely "lazy" in that it does not immediately perform the work that a given map / etc. might invoke on all the data in an array the same way that a map might work on e.g. a list.
2. While the library provides a storage-class-like interface, it does not always make it easy to pull an FFI-compatible structure out from the "array" (again, due to the "lazy" API).

The largest reason of which being the first: the laziness invoked by SRFI 231's API. It ends up muddying the distinction between intervals and arrays, and behaves very differently than what many Scheme libraries would leave one to expect. The laziness has a purpose in SRFI 231: it serves to avoid making lots of expensive intermediate copies of arrays in memory as operations stack up. The latter reason listed above is a downstream effect of the laziness in SRFI 231, which makes it sometimes awkward to use.

Avoiding expensive intermediate copies of large arrays is important; however, this is better solved by composing operations over a transducer than by letting that concern bleed through the entire API. What's more, intervals (as provided by this library, not SRFI 231) combined with transducers are useful unto themselves. The "basic" implementation of indices-as-vectors likewise leads to a pleasantly expressive (and fast) way to quickly loop over multiple dimensions.

In the end, I chose to depart from implementing as many procedures as SRFI 231, in particular avoiding:

• One-off whole-array transformations, such as array-map, array-any, array-every, etc.
• Conversions to other data structures such as vectors and lists.* A conversion array->nested-list is provided purely for providing a pretty-printer for array objects
• Inner and outer products

The first two points above are a consequence of making the arrays API transducer-aware. The latter is a conscious culling of functionality. Quite bluntly: no direct Scheme API will ever perform large-scale mathematics (such as matrix operations) efficiently, nor will it be easy to optimize. What is more important than providing a variety of poorly-optimized mathematical operations is to provide a data structure that enables ease of construction and manipulation, as well as easy access to ways in which the array can be conveniently passed to some foreign function interface (FFI) so as to allow a lower-level mathematical language (C, Fortran, Rust) to run much more tightly optimized mathematical code on the array.

It is under that belief that I say this: "If you're using generalized-arrays in the hope that you're going to implement BLAS/LAPACK, or Eigen3, or SuiteSparse, be warned that you'd be better served wrapping those libraries in your favourite Scheme's FFI and instead passing the underlying array to them." Full stop.

For that reason, generalized-arrays is not a great library for doing math. But if you need to store, manipulate, print, slice, or traverse an array that you plan to send to some other library that does math, you're in the right place.

API Reference

The below API reference is best understood in conjunction with transducers. If you are unsure of any of the examples, it is strongly suggested that you start by understanding how transducers work first.

(generalized-arrays intervals)

<interval>record

A record type that represents a multi-dimensional, half-open interval. It is described by two Scheme vectors, the interval-start and interval-end, which represent the lower and upper bounds of the interval, respectively.

vector-index-comparatorconstant

A SRFI-128 comparator that is meant for comparing Scheme vector indices in an interval.

check-interval loc arg arg-nameprocedure

Checks if the interval arg is an interval. If it is not, signals an error that describes that arg is not an interval in the function with the name loc (a symbol) and name arg-name (also a symbol).

 
(import (generalized-arrays intervals))

(check-interval 'my-procedure 3 'interval)

; => Error: (my-procedure) bad `interval` argument type - not an interval: 3

make-interval start endprocedure

Constructs a new half-open interval from the start index up to but not including the end index. Both indices must be vectors of the same length (rank). If any dimension inside the start index is greater than or equal to its equivalent dimension in the end index, then the interval is empty.

Indices in this library cannot contain dimension values less than zero.

make-default-interval endprocedure

Constructs a default interval from the provided end index. The "start" index for this interval is an index with the same rank as the end index, but where all the dimensions are equal to zero.

interval? iprocedure

A predicate for evaluating if an object is an interval. True iff the interval is an <interval> record.

interval-start iprocedure

interval-end iprocedure

Procedures that grab the start and end indices of an interval, respectively.

interval-length iprocedure

Procedure that computes the total number of elements in the interval. This is equivalent to taking the lengths along each dimension and multiplying them.

 
(import (generalized-arrays intervals)
        (test))

(test "Length of [(0, 0) (2, 2)) is 4"
       4
       (make-default-interval (vector 2 2)))

(test "Length of [(1, 1) (3, 4)) is 6"
       6
       (make-interval (vector 1 1)
                      (vector 3 4)))

interval-empty? iprocedure

Predicate which returns true iff the interval i has a length of zero.

interval-contains? interval idxprocedure

A predicate which returns true iff the index idx is contained within the interval.

 
(import (generalized-arrays intervals)
        (test))

(test-assert "Index #(1 1) is contained within [(0, 0) (3, 3))"
             (interval-contains? (make-default-interval (vector 3 3))
                                 (vector 1 1)))

interval-fold f sentinel xsprocedure

reverse-interval-fold f sentinel xsprocedure

Transducer-aware folding operations over intervals (forward and reverse).

 
(import (generalized-arrays intervals)
        (transducers)
        (test))

(test "Transducing over interval values"
      (list (vector 0 0)
            (vector 0 1)
            (vector 1 0)
            (vector 1 1))
      (transduce interval-fold
                 values
                 (collect-list)
                 (make-default-interval (vector 2 2))))

(test "Transducing over interval values in reverse"
      (list (vector 1 1)
            (vector 1 0)
            (vector 0 1)
            (vector 0 0))
      (transduce reverse-interval-fold
                 values
                 (collect-list)
                 (make-default-interval (vector 2 2))))

flatten-intervalprocedure

reverse-flatten-intervalprocedure

Transducer that flattens transduced intervals either in forward or reverse order.

 
(import (generalized-arrays intervals)
        (transducers)
        (test))

(test "Flattening intervals constructed by a separate transducer procedure"
      (list (vector 0 0)
            (vector 0 1)
            (vector 1 0)
            (vector 1 1)
            (vector 5 5)
            (vector 5 6)
            (vector 6 5)
            (vector 6 6))
      (transduce list-fold
                 (compose
                   (zip-list (list (vector 2 2) (vector 7 7)))
                   (map (lambda (pair) (apply make-interval pair)))
                   flatten-interval)
                 (collect-list)
                 (list (vector 0 0) (vector 5 5))))

chain-interval intervalprocedure

reverse-chain-interval intervalprocedure

Transducers that chain the indices contained in the intervals to the transduction operation, in either forward or reverse order.

 
(import (generalized-arrays intervals)
        (transducers)
        (test))

(test "Chaining an interval to a list of symbols"
      (list 'a
            'b
            'c
            'd
            (vector 0 0)
            (vector 0 1)
            (vector 1 0)
            (vector 1 1))
      (transduce list-fold
                 (chain-interval
                   (make-default-interval (vector 2 2)))
                 (collect-list)
                 (list 'a 'b 'c 'd)))

zip-interval intervalprocedure

reverse-zip-interval intervalprocedure

Transducers that zip the indices contained within the interval to another transduction, in forward or reverse order.

 
(import (generalized-arrays intervals)
        (transducers)
        (test))

(test "Zipping an interval onto elements of a list"
      (list (cons 'a (vector 0 0))
            (cons 'b (vector 0 1))
            (cons 'c (vector 1 0))
            (cons 'd (vector 1 1)))
      (transduce list-fold
                 (zip-interval
                   (make-default-interval (vector 2 2)))
                 (collect-list)
                 (list 'a 'b 'c 'd 'e)))

interleave-interval intervalprocedure

reverse-interleave-interval intervalprocedure

Transducers that interleave the indeices contained within the interval with another transduction, in forward or reverse order.

 
(import (generalized-arrays intervals)
        (transducers)
        (test))

(test "Interleaving an interval onto elements of a list"
      (list 'a (vector 0 0)
            'b (vector 0 1)
            'c (vector 1 0)
            'd (vector 1 1))
      (transduce list-fold
                 (zip-interval
                   (make-default-interval (vector 2 2)))
                 (collect-list)
                 (list 'a 'b 'c 'd 'e)))

(generalized-arrays storage)

Storage classes are a type-class like object that holds procedures for working with storage objects. Storage classes themselves do not hold data; however, they can be used to construct a storage object (e.g. a vector, an f64vector) that can hold data. These storage objects can be manipulated by the corresponding procedures for a given storage class.

<storage-class>record

A unique record type that denotes an object which is a storage-class.

make-storage-class short-id constructor ref set length copy copy! transducible comparator default-elementprocedure

Constructor for creating a new storage class. See the following accessors for more information about what each of the individual elements should be.

storage-class? scprocedure

A type predicate which returns true iff the provided object is a storage-class.

storage-class-short-id scprocedure

A short ID for the storage class sc. This identifier exists to be a unique short-code for a storage class' type or kind. For example, a vector-storage-class (which uses a Scheme vector as the underlying storage type) would have a short ID that is the symbol 'v. This is primarily used for reader and writer syntax.

storage-class-constructor scprocedure

A procedure of the form (make-storage size #!optional fill) which constructs a new storage object of the implementing storage class sc.

storage-class-ref scprocedure

A procedure of the form (storage-ref storage index) which gets an individual element at index from the storage object of the implementing storage class sc.

storage-class-set scprocedure

A procedure of the form (storage-set! storage index value) which sets value to the element at index in storage object of the implementing storage class sc.

storage-class-length scprocedure

A procedure of the form (storage-length storage) which gets the number of total elements in the storage object of the implementing storage class sc.

storage-class-copy scprocedure

A procedure of the form (storage-copy storage) which creates a direct copy of the storage object of the implementing storage class sc and returns it.

storage-class-copy! scprocedure

A procedure of the form (storage-copy storage) which creates a direct copy of the storage object of the implementing storage class sc and returns it.

storage-class-transducible scprocedure

A transducible type-class over the storage object kind. See the documentation of transducible type-classes in the transducers library for more details.

storage-class-comparator scprocedure

A SRFI-128 comparator for storage objects of the storage class sc.

storage-class-default-element scprocedure

A value which represents the default element of the storage object. This is used when producing a default storage object from storage class sc, or filling individual elements with a default value when filtered.

make-storage-object storage-class n #!optional fillprocedure

Constructs a storage object of the provided storage-class with n elements defaulting to the provided fill value. If fill is not provided, the storage-class` default element will be used instead.

storage-object-ref storage-class storage-object iprocedure

Gets the ith element contained inside the storage-object.

storage-object-set! storage-class storage-object i valueprocedure

Sets the ith element contained inside the storage-object to value.

storage-object-length storage-class storage-objectprocedure

Gets the length of the storage-object.

storage-object-copy storage-class storage-object start #!optional endprocedure

Constructs a new storage-object of length (- end start) and copies the values from storage-object into this newly constructed object.

storage-object-copy! to at from #!optional start endprocedure

Copies the values in storage-object from into storage-object to starting at index at. The values copied are from start to end (length (- end start)) in the from object.

vector-storage-classconstant

u8vector-storage-classconstant

s8vector-storage-classconstant

u16vector-storage-classconstant

s16vector-storage-classconstant

u32vector-storage-classconstant

s32vector-storage-classconstant

u64vector-storage-classconstant

s64vector-storage-classconstant

f32vector-storage-classconstant

f64vector-storage-classconstant

c64vector-storage-classconstant

c128vector-storage-classconstant

Standard, preconfigured storage classes. Each storage class uses the specific "vector" Scheme type (either standard Scheme vectors or SRFI-4 / SRFI-160 vectors) as a backing storage, as specified by its name.

(generalized-arrays arrays)

<array>record

A unique record type for representing multi-dimensional array data.

array? objprocedure

A predicate procedure that returns #t iff the object is an array (of type <array>), otherwise #f.

array-view? arrayprocedure

Predicate which evaluates to #t iff the array is some kind of array view, otherwise #f. An array can be a slice, or any array that has had its axes re-arranged in some way (tranposition, swap-axes, squeeze, expand), while maintaining the same underlying storage object and interval rank.

make-array storage-class dimension #!optional fillprocedure

Constructs an array by allocating a storage object of the provided storage-class and dimension. If fill is provided, every value in the array is set to fill, otherwise the array is defaulted with the value as if calling (storage-class-default-element storage-class).

 
(import generalized-arrays
        test)

(test-assert "Array is filled with #f"
  (array=? #a2v((#f #f #f)
                (#f #f #f)
                (#f #f #f))
           (make-array vector-storage-class #(3 3) #f)))

make-array-from-storage storage-class dimension storage-objectprocedure

Constructs an array of the provided storage-class and dimension with a provided storage-object. Unlike make-array, this does not allocate any storage object on its own. Additionally, it is an error if:

• The storage-object is not a storage object of the provided storage-class
• The length of the interval constructed by (make-default-interval dimension) is not exactly equal to the length of the storage-object.

This procedure is helpful if you are working with array data via SRFI-4 and / or SRFI-160 storage kinds but are constructing or managing the storage across the FFI boundary.

 
(import generalized-arrays
        test)

(define vec #f64(1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0))

(define arr
  (make-array-from-storage
    f64vector-storage-class #(3 3) vec))

(test "Storage object in array is equal to original f64vector"
      vec
      (array-storage-object arr))

array-tabulate storage-class dimension procprocedure

Constructs an array by allocating a storage-object of the provided storage-class with the provided dimension(s) as make-array does. Unlike make-array which fills the array with a single value, array-tabulate sets the value at each index by calling (proc index) for every index in the array.

 
(import generalized-arrays
        test)

(test "Constructed array contains all of its indices."
      #a2v((#(0 0) #(0 1) #(0 2))
           (#(1 0) #(1 1) #(1 2))
           (#(2 0) #(2 1) #(2 2)))
      (array-tabulate vector-storage-class
                      #(3 3)
                      (lambda (idx) idx)))

array-broadcast array valueprocedure

Constructs a new array by copying the storage-class and shape of the input array and setting all the elements of the newly allocated array to the provided value.

 
(import generalized-arrays
        test)

(define a (make-array vector-storage-class #(3 3) 0))
(define b (array-broadcast a "abc"))

(test "Shapes of arrays are the same."
      (array-shape a)
      (array-shape b))

(test "Storage class of arrays are the same."
      (array-storage-class a)
      (array-storage-class b))

array-storage-class arrayprocedure

Returns the storage-class of the provided array.

array-interval arrayprocedure

Returns the "interval" of a given array. The current interval of a given array is the range of multi-dimensional indices that are valid for that array object. It is not always the case that the interval starts at the default (zero) multi-index. For example:

 
(import generalized-arrays)

(define a
  (make-array vector-storage-class
              #(3 3)
              0))

(test "Interval of array a"
      (make-interval #(0 0) #(3 3))
      (array-interval a))

(define b
  (array-slice a #(1 1) #(2 2)))

(test "Interval of array b"
      (make-interval #(1 1) #(2 2))
      (array-interval b))

In the above example, the array b is a slice of array a, from #(1 1) to #(2 2), so while b shares the same storage-object as a, the interval of b is smaller than that of a.

array-interval is meant to be used as a low-level mechanism for being able to work with slices over a shared storage-object. In most cases, you'll want to use array-shape to get the semantic interval over the array, rather than array-interval.

array-shape arrayprocedure

Returns the shape of a given array. The shape of an array is the default interval up to the maximum length of each dimension of the array or view. Unlike array-interval, the shape of an array always starts at the default (zero) index, and ends at the maximum dimension of the array.

In almost all instances, array-shape is the way one should get the full interval of indices in a given array. While array-interval provides the actual interval used to index into the storage-object of the array, array-shape is more useful when using most array APIs (e.g. array-ref); of which, these APIs will expect an array that is in the bounds of the shape rather than the interval, which is meant to be a tool for developers who are operating on the raw, underlying array structure directly.

 
(import generalized-arrays)

(define a
  (make-array vector-storage-class
              #(3 3)
              0))

(test "Shape of array a"
      (make-interval #(0 0) #(3 3))
      (array-shape a))

(define b
  (array-slice a #(1 1) #(2 3)))

(test "Interval of array b"
      (make-interval #(1 1) #(2 3))
      (array-interval b))

(test "Shape of array b"
      (make-interval #(0 0) #(1 2))
      (array-shape b))

array-storage-object arrayprocedure

Gets the underlying storage-object of the array. This is primarily meant if one wants to get the underlying storage in order to pass it to some specialized procedure that can operate on the array contents directly. An example of this could be passing the underlying storage object to the FFI, such as to use BLAS or LAPACK optimized procedures over the underlying array storage.

For example, see the following example (copied from the BLAS egg):

 
(import generalized-arrays
        blas
        srfi-4
        test)

(define order RowMajor)
(define transa NoTrans)

(define m 4)
(define n 4)

(define alpha 1)
(define beta 0)

(define a
  (make-array-from-storage
    f64vector-storage-class
    (vector m n)
    (f64vector 1 2 3 4
               1 1 1 1
               3 4 5 6
               5 6 7 8)))

(define x (f64vector 1 2 1 1))
(define y (f64vector 0 0 0 0))

(dgemv! order transa m n alpha (array-storage-object a) x beta y)

(test "Result of multiplying the array a by the vector x"
      y
      (f64vector 12 5 22 32))

Generally speaking, the above is a contrived example; However, this contrived example demonstrates how one might start to build a more complete (and optimized) mathematics library that can be utilized in conjunction with the provided arrays API.

array-rank arrayprocedure

Returns the rank (number of dimensions) of the provided array.

 
(import generalized-arrays
        test)

(test "Rank of array is 2"
      2
      (make-array vector-storage-class #(2 2)))

(test "Rank of array is 3"
      3
      (make-array vector-storage-class #(2 3 3)))

array-strideprocedure

Gets the stride of each dimension in the array.

array-ref array idxprocedure

Gets the element at idx in the array.

 
(import generalized-arrays
        test)

(define a
  (make-array-from-storage
    vector-storage-class
    (vector 3 3)
    (vector 1 2 3
            4 5 6
            7 8 9)))

(test "Element at #(1 1) is equal to 5"
      5
      (array-ref a #(1 1)))

array-set! array idx valueprocedure

Sets the element in array at idx to the provided value. Analagous to something like vector-set!, but for arrays.

 
(import generalized-arrays
        test)

(define a
  (make-array-from-storage
    vector-storage-class
    (vector 3 3)
    (vector 1 2 3
            4 5 6
            7 8 9)))

(test "Element at #(1 1) is equal to 5"
      5
      (array-ref a #(1 1)))

(array-set! a #(1 1) "abc")

(test "Element at #(1 1) is now equal to 'abc'"
      "abc"
      (array-ref a #(1 1)))

array-update! array idx procprocedure

Updates the element in the array at idx as if applying proc to it.

 
(import generalized-arrays
        test)

(define a
  (make-array-from-storage
    vector-storage-class
    (vector 3 3)
    (vector 1 2 3
            4 5 6
            7 8 9)))

(test "Element at #(1 1) is equal to 5"
      5
      (array-ref a #(1 1)))

(array-update!
  a
  #(1 1)
  (lambda (elem) (- elem 20)))

(test "Element at #(1 1) is now equal to -15"
      -15
      (array-ref a #(1 1)))

array=? lhs rhsprocedure

A predicate function which evalutes if the dimensions of the left and right arrays (lhs and rhs respectively) are equal, and all of the elements at each index position in the left and right arrays are equal as if applying equal?.

This is mostly useful in writing tests, and is unlikely to be too useful outside of them.

array-fold f sentinel xsprocedure

reverse-array-fold f sentinel xsprocedure

Transducer-aware folding operations over arrays. Every element in the array is folded in lexicographic order, starting at the default (zero) index of the array and ending at the maximum dimension of the array.

 
(import generalized-arrays
        transducers
        test)

(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 3)
                           (vector 1 2 3
                                   4 5 6
                                   7 8 9)))

(test "Sum of all array values is 45"
      45
      (transduce array-fold
                 values
                 (collect-sum 0)
                 a)

array-reverse-fold works like array-fold, but folds over the array in reverse-lexicographic order, starting at the maximum dimension of the array and ending at the default (zero) index of the array.

array-interval-fold f sentinel xsprocedure

reverse-array-interval-fold f sentinel xsprocedure

Transducer-aware folding operations that serve a convenience to fold over all the indices in an array. This can be used as a shorthand form of calling interval-fold over the (array-shape array).

flatten-arrayprocedure

reverse-flatten-arrayprocedure

Transducers that flatten arrays into their contained values in a transduction. flatten-array flattens the elements of the array into the transduction in lexicographic ordering. reverse-flatten-array is similar to flatten-array, but flattens the values in reverse-lexicographic ordering.

 
(import generalized-arrays
        transducers
        test)

(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 3)
                           (vector 1 2 3
                                   4 5 6
                                   7 8 9)))

(test "Flattening a list of arrays"
      (vector 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9)
      (transduce list-fold
                 flatten-array
                 (collect-vector)
                 (list a a)))

chain-array arrayprocedure

reverse-chain-array arrayprocedure

Transducer that chains the contents of the provided array onto the end of the transduction. reverse-chain-array works similar to chain-array, but chains the array onto the transduction in reverse-lexicographic ordering.

 
(import generalized-arrays
        transducers
        test)

(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 3)
                           (vector 'a 'b 'c
                                   'd 'e 'f
                                   'g 'h 'i)))

(test "Chaining array to end of list-transduction"
      (list 1 2 3 'a 'b 'c 'd 'e 'f 'g 'h 'i)
      (transduce list-fold
                 (chain-array a)
                 (collect-list)
                 (list 1 2 3)))

zip-array arrayprocedure

reverse-zip-array arrayprocedure

Transducer that zips the contents of the provided array onto the transduction. reverse-zip-array works similar to zip-array, but zips the array onto the transduction in reverse-lexicographic ordering.

 
(import generalized-arrays
        transducers
        test)

(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 3)
                           (vector 'a 'b 'c
                                   'd 'e 'f
                                   'g 'h 'i)))

(test "Zip array to transduction"
      (list (cons 1 'a) (cons 2 'b) (cons 3 'c))
      (transduce list-fold
                 (zip-array a)
                 (collect-list)
                 (list 1 2 3))))

(test "Reverse zip array to transduction"
      (list (cons 1 'i) (cons 2 'h) (cons 3 'g))
      (transduce list-fold
                 (reverse-zip-array a)
                 (collect-list)
                 (list 1 2 3)))

interleave-array arrayprocedure

reverse-interleave-array arrayprocedure

Transducer that interleaves the contents of the provided array into the transduction. reverse-interleave-array works similar to interleave-array, but interleaves the array onto the transduction in reverse-lexicographic ordering.

 
(import generalized-arrays
        transducers
        test)

(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 3)
                           (vector 'a 'b 'c
                                   'd 'e 'f
                                   'g 'h 'i)))

(test "Interleave array to transduction"
      (list 1 'a 2 'b 3 'c)
      (transduce list-fold
                 (interleave-array a)
                 (collect-list)
                 (list 1 2 3))))

(test "Reverse zip array to transduction"
      (list 1 'i 2 'h 3 'g)
      (transduce list-fold
                 (reverse-interleave-array a)
                 (collect-list)
                 (list 1 2 3)))

collect-array storage-class dimensionprocedure

A transducer-collector (reducer) used to collect values into an array of the provided storage-class and dimension. The number of elements collected must be exactly equal to the length of the default-interval generated from the provided dimension, or an error is signaled.

array-transducibleconstant

reverse-array-transducibleconstant

Transducible type-classes over the lexicographic and reverse-lexicographic transducer operations over arrays.

array-slice array start #!optional endprocedure

Slices an array starting from the start index to the (optional) end index. This procedure produces an array view (that satisfies array-view?) representing the sub-interval defined by [start end) in the array.

This procedure doesn't copy any elements of the array or modify the internal storage object shared with the original array, which makes it useful for designating smaller sub-sections of the array, for e.g. transduction or mutation.

 
(import generalized-arrays
        test)

(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 3)
                           (vector 'a 'b 'c
                                   'd 'e 'f
                                   'g 'h 'i)))

;; Slice the array
;;
;; By default, `end` is the end of the array `a`
(define b (array-slice a #(1 1)))

(test-assert "Slice of array a"
  (array=? b (make-array-from-storage vector-storage-class
                                      (vector 2 2)
                                      (vector 'e 'f
                                              'h 'i)))

array-transpose arrayprocedure

Performs a monadic transposition of the major through minor axes of the array. This is equivalent to reversing the ordering of the major through minor axes. This procedure returns an array view (i.e. satisfies array-view?) without copying any of the array's internal data or modifying the underlying storage-object.

 
(import generalized-arrays
        test)

(define a
  (make-array vector-storage-class
              (vector 1 2 3 4)
              0))

(test "Array dimensions are (1 2 3 4)"
      (make-default-interval (vector 1 2 3 4))
      (array-shape a))

(define a-t
  (array-transpose a))

;; For a rank-2 array, this would be swapping an #(i j) with #(j i)
(test "Transpose dimensions are (4 3 2 1)"
      (make-default-interval (vector 1 2 3 4))
      (array-shape a-t))

Note that while the internal storage-object is not modified, fold operations still iterate through the array's elements (specifically a-t in the example above) in lexicographic order according to the array's shape / interval. This may or may not provide efficient access into the storage-object, since sequential elements in lexicographic order may not necessarily be contiguous in memory.

array-swap-axes array to fromprocedure

Perform a triadic transposition of the to and from axes. This is equivalent to swapping the ordering of the two provided axes in the array's shape. This procedure returns an array view (i.e. satisfies array-view?) without copying any of the array's internal data or modifying the underlying storage-object.

Both the to and from axes must be fixnums between 0 and the less than the rank of the array.

 
(import generalized-arrays
        test)

(define a
  (make-array vector-storage-class #(2 1 3 4) 0))

(test "Shape of array a"
      (make-default-interval (vector 2 1 3 4))
      (array-shape a))

(define b
  (array-swap-axes a 0 1))


(test "Shape of array b"
      (make-default-interval (vector 1 2 3 4))
      (array-shape b))

(define c
  (array-swap-axes b 0 3))

(test "Shape of array c"
      (make-default-interval (vector 4 2 3 1))
      (array-shape b))

(define d
  (array-swap-axes a 2 3))

(test "Shape of array d"
      (make-default-interval (vector 2 1 4 3))
      (array-shape d))

array-squeeze-axis array axisprocedure

Removes the axis from the array's shape, if and only if the length of that axis is 1. If the length of that axis is not 1, an error is signalled. This procedure returns an array view (i.e. satisfies array-view?) without copying any of the array's internal data or modifying the underlying storage-object.

 
(import generalized-arrays
        test)

(define a (make-shape vector-storage-class (vector 2 1 2) 0))

(test "Squeezing array axis"
      (make-default-interval (vector 2 2))
      (array-shape
        (array-squeeze-axis a 1)))

(test-error "Squeezing an incorrect axis"
            (array-squeeze-axis a 0))

array-squeeze arrayprocedure

Removes all axes in the array's shape whose length is 1. If every axis would be squeezed by this operation, returns the singular value in the array. This procedure returns an array view (i.e. satisfies array-view?) without copying any of the array's internal data or modifying the underlying storage-object.

 
(import generalized-arrays
        test)

(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 1 3)
                           (vector 'a 'b 'c
                                   'd 'e 'f
                                   'g 'h 'i)))

(test "Squeeze array axes"
      (make-default-interval (vector 3 3))
      (array-shape
        (array-squeeze a)))

(test "Get single value from slice"
      'e
      (array-squeeze
        (array-slice a #(1 0 1) #(2 1 2))))

array-expand-axis array axisprocedure

Adds an axis of length 1 before axis to the internal shape of the array. This procedure returns an array view (i.e. satisfies array-view?) without copying any of the array's internal data or modifying the underlying storage-object.

This procedure can be though of as the opposite of array-squeeze-axis.

 
(import generalized-arrays
        test)
(define a
  (make-array-from-storage vector-storage-class
                           (vector 3 3)
                           (vector 'a 'b 'c
                                   'd 'e 'f
                                   'g 'h 'i)))

(test "Shape of expanded array"
      (make-default-interval (vector 1 3 3))
      (array-shape
        (array-expand-axis a 0)))

array-copy array #!optional start endprocedure

Makes a copy of the array by allocating a new storage object, with the values copied into the new storage-object in lexicographic order.

This procedure is often most useful when one wants to pass the storage object of an array to some lower-level API (e.g. BLAS). Array views (i.e. satisfies array-view?) share the underlying storage object with an array allocated previously, and so need to be coerced into a newly allocated copy of the storage object if one wants to use the storage object with e.g. BLAS.

 
(import generalized-arrays
        blas
        srfi-4
        test)

(define order ColMajor)
(define transa NoTrans)

(define alpha 1)
(define beta 0)

(define a
  (make-array-from-storage
    f64vector-storage-class
    (vector 5 5)
    ;; Padded zeros to demonstrate how slicing might be wrong
    (f64vector 0 0 0 0 0
               0 1 1 3 5
               0 2 1 4 6
               0 3 1 5 7
               0 4 1 6 8)))

(define x (f64vector 1 2 1 1))
(define y (f64vector 0 0 0 0))

(define b
  (array-transpose
    (array-slice a #(1 1))))

;; WRONG! Do not do the following, since the storage object of `b` is the same
;; storage object of `a`. Instead, we need to first define a copy `c` that
;; copies the data so that the storage object has the correct layout.
;;
;; (dgemv! order transa 4 4 alpha (array-storage-object b) x beta y)

(define c (array-copy b))
(dgemv! order transa 4 4 alpha (array-storage-object a) x beta y)

(test "Result of multiplying the array c by the vector x"
      y
      (f64vector 12 5 22 32))

array-copy! to at from #!optional start endprocedure

Copies the elements of array from into the array to starting at index at. If start and end are provided, these specify the array interval inside from to copy across. An error is signaled if the shapes of the array to and the interval specified by start and end are incongruent.

 
(import generalized-arrays
        test)

(define a
  (make-array vector-storage-class #(3 3) 0))

(define b
  (make-array vector-storage-class #(4 4) 1))

(array-copy! (array-slice a #(1 1) #(3 3))
             (vector 0 0)
             b)

(test-assert "Array a has had its lower corner copied from b"
  (array=? a
           (make-array-from-storage vector-storage-class
                                    #(3 3)
                                    (vector 0 0 0
                                            0 1 1
                                            0 1 1))))

array-append axis arr brrprocedure

Appends the array brr to the array arr along the provided axis. An error is signaled if the arrays have different ranks, if the provided axis is less than zero or greater than the rank of the arrays, or if the axes outside of axis are not equal.

 
(import generalized-arrays
        test)

(define a
  (make-array vector-storage-class #(2 2) 0))

(define b
  (make-array vector-storage-class #(2 2) 1))

(define c
  (array-append 0 a b))

(test-assert "Appending arrays a and b produces 4x2 array"
  (array=? c
           (make-array-from-storage vector-storage-class
                                    #(4 2)
                                    (vector 0 0
                                            0 0
                                            1 1
                                            1 1))))

array-reshape array dimensionprocedure

Reshapes the array to the default interval constructed by the provided dimension. This is done by performing a full copy of the array (as opposed to creating an array view). An error is signaled if the new shape does not have the same length as the existing shape.

 
(import generalized-arrays
        test)

(define a
  (make-array-from-storage vector-storage-class
                           #(4)
                           (vector 1 2 3 4)))

(test-assert "Reshaping array to a 2x2"
  (array=? (array-reshape a #(2 2))
           (make-array-from-storage vector-storage-class
                                    #(2 2)
                                    (vector 1 2
                                            3 4))))

(test-assert "Array-reshape never makes an array view"
  (not (array-view? (array-reshape a #(2 2)))))

array-reclassify array storage-classprocedure

Reclassifies the storage of an array by copying the data to a newly constructed array and storage-object of the provided storage-class.

 
(import generalized-arrays
        srfi-4
        test)

(define a
  (make-array vector-storage-class #(3 3) 0))

;; Reclassify into f32vector storage
(define b
  (array-reclassify a f32vector-storage-class))

(test-assert "Original array holds vector storage object"
  (vector? (array-storage-object b)))

(test-assert "Reclassified array holds f32vector storage object"
  (f32vector? (array-storage-object b)))

array->nested-list arrayprocedure

Converts an array's data into a nested-list representation. This API is mostly not intended for broad use and is only provided for writing data out in a structured manner.

array-read #!optional portprocedure

Reads an array (similar to read) from the provided port, or (current-input-port) if port is not specified. The array is read in using the optional reader syntax provided with this library:

 
;; #a for array
;; 2 for rank 2
;; v for vector-storage-class (short-id of the storage-class)
#a2v((1 2 3)
     (4 5 6)
     (7 8 9))

Note that if the storage class is unknown (unknown short-id) then the procedure will default to reading in the array with a vector-storage-class.

array-write #!optional portprocedure

Writes an array (similar to write) to the provided port, or (current-output-port) if port is not specified. The array is written out using the optional reader syntax provided with this library:

 
;; #a for array
;; 2 for rank 2
;; v for vector-storage-class (short-id of the storage-class)
#a2v((1 2 3)
     (4 5 6)
     (7 8 9))

License

The code in this library is distributed according to the 3-clause BSD license. See LICENSE for more details.