chickadee » skiplists

Skiplists

Skiplists are data-types, which can replace balanced search-trees. They are invented by Pugh. The idea is as follows:

Contrary to listnodes, which are pairs of an item and a next pointer, skipnodes are pairs of an item and a vector of next pointers. The length' of these vectors depend on each skipnode itself. They are choosen randomly in such a way, that, in the average, the number of nodes with at least k links is half the number of links with at least k-1 links, for k>1. Following the next pointers at a fixed link-level, k say, one skips all nodes with less than k pointers.

Inserting an item into a skiplist now works as follows. First one packages the item into a skipnode, where the number of links is generated randomly as described above. Second, one follows the skiplist along the highest occupied number of links as long as the skiplist's nodes point to items less then the item of the node to be inserted. Third, one steps down one level and continues following the skiplist's nodes at this new smaller level. Repeating this process until level 0 is reached we eventually find the place where our new node is to be inserted.

Some additional remarks are in order.

We described the process with a width of two, i.e. at each level in the average one node of the level below is skipped. A higher value than two for the width is possible as well, trading performance against space.

We have to decide, what to do with duplicates. We choose the following approach: The skiplist itself stores a list of either one or several numerical comparison operators. Only if the last of those operators is the special comparison operator dups (which returns constantly 0, i.e. it compares nothing) duplicates are allowed. Moreover, we arrage matters in such a way, that all nodes of duplicates with the same key have the same height, so that a search for the item which was inserted last will be found first.

Documentation

In this implementation skiplists are implemented in two modules, %skiplists and skiplists. They both have the same interface. The former implements all routines without any checks, the latter imports the former with routines prefixed with % and calling those routines with checked arguments. This way you can trade security against speed ...

skiplists

skiplistsprocedure

skiplists symprocedure

documentation procedure. The first call shows the list of exported symbols, the second documentation of symbol sym.

skiplist

skiplist width max-height item? order #!rest ordersprocedure

skiplist max-height item? order #!rest ordersprocedure

skiplist item? order #!rest ordersprocedure

constructors: width is the jump width, max-height the maximum allowed length of pointers of an item, item? checks

skiplist?

skiplist? xprprocedure

type predicate.

skiplist->list

skiplist->list slsprocedure

skiplist->list sls levelprocedure

the list of items stored in each level

sl-null?

sl-null? slsprocedure

is skiplist empty?

sl-dups?

sl-dups? slsprocedure

are duplicates allowed?

sl-item?

sl-item? slsprocedure

item type predicate

dups

dups x yprocedure

trivial numerical comparison operator to be used as last order to allow duplicates.

transformer.

sl-compare

sl-compare slsprocedure

combined comparison function

sl-count

sl-count slsprocedure

number of items stored in skiplist

sl-found

sl-found slsprocedure

list of found items, to be called after search!

sl-found?

sl-found? sls itemprocedure

is item found?

sl-height

sl-height slsprocedure

actual maximal height of nodes (can be changed)

sl-max-height

sl-max-height slsprocedure

absolute maximum heigth of nodes in skiplist (not changeble)

sl-max

sl-max slsprocedure

biggest item stored in skiplist

sl-min

sl-min slsprocedure

smallest item stored in skiplist

sl-orders

sl-orders slsprocedure

list of orders defined in the constructor

sl-search-level

sl-search-level slsprocedure

down to which level a previous search descended?

sl-width

sl-width slsprocedure

width skipped on average at each search level supplied by constructor

sl-map

sl-map fn slsprocedure

sl-map fn sls order #!rest ordersprocedure

sl-map fn sls widthprocedure

sl-map fn sls width order #!rest ordersprocedure

depending on the mapping function, different order procedures might be necessary

sl-for-each

sl-for-each proc slsprocedure

apply proc to each item in skiplist

sl-filter

sl-filter ok? slsprocedure

filtering

sl-reorder

sl-reorder sls order #!rest ordersprocedure

changing orders

sl-restructure

sl-restructure sls width max-heightprocedure

changing width

sl-insert!

sl-insert! sls item #!rest itemsprocedure

insert items into skiplist

sl-remove!

sl-remove! sls item #!rest itemsprocedure

remove items from skiplist

sl-search!

sl-search! sls itemprocedure

searching for an item changes internal cursor transparently

sl-clear!

sl-clear! slsprocedure

reset skiplist

Examples

(import skiplists)

   A skiplist with primary and secondary search order, allowing duplicates
;; some constructors

(define sls1 (skiplist 15 fixnum? -))
(sl-width sls1) ;-> 2
(sl-max-height sls1) ;-> 15
(sl-dups? sls1) ;-> #f

(define sls2 (skiplist 4 20 fixnum? - dups))
(fx= (sl-width sls2) ;-> 4
(fx= (sl-max-height sls2) ;-> 20
(sl-dups? sls2) ;-> #t

;; create ...

(define item-type (lambda (x)
                    (and ((list-of? integer?) x) (> (length x) 2))))

(define primary-order (lambda (x y) (- (car x) (car y))))

(define secondary-order (lambda (x y) (- (cadr x) (cadr y))))

(define sls3 (skiplist 3
                       15
                       item-type
                       primary-order
                       secondary-order
                       dups))

;; and populate ...

(define lst1
        (let loop ((k 0) (lst '()))
          (if (= k 100)
            lst
            (loop (+ k 1) (cons (pseudo-random-integer 10) lst)))))

(define lst2
        (let loop ((k 0) (lst '()))
          (if (= k 100)
            lst
            (loop (+ k 1) (cons (pseudo-random-integer 10) lst)))))

(define lst3
        (let loop ((k 0) (lst '()))
          (if (= k 100)
            lst
            (loop (+ k 1) (cons (pseudo-random-integer 100) lst)))))

(apply sl-insert! sls3
       (map (lambda (x y z) (list x y z))
            lst1 lst2 lst3))

(sl-count sls3) ;-> 100

(sl-width sls3) ;-> 3

;; inserting item and removing all with same key ...

((sl-item? sls3) '(1 2 3)) ;-> #t

(sl-search! sls3 '(1 2 3))

(if (sl-found? sls3 '(1 2 3))
  (apply sl-remove! sls3 (sl-found sls3)))

(sl-insert! sls3 '(1 2 3))

(sl-search! sls3 '(1 2 3))

(member '(1 2 3) (sl-found sls3))

(apply sl-remove! sls3 (sl-found sls3))

(sl-search! sls3 '(1 2 3))

(null? (sl-found sls3))

;; produce duplicates at the ends ...

(sl-insert! sls3 '(-1 2 3) '(-1 2 3 4))

(sl-min sls3) ;-> '((-1 2 3 4) (-1 2 3))

(sl-insert! sls3 '(10 1 2) '(10 1 2 3) '(10 1 3))

(sl-found sls3) ;-> '((10 1 3) (10 1 2 3) (10 1 2))

(sl-max sls3) ;-> '((10 1 3) (10 1 2 3) (10 1 2))

;; and remove them again ...

(sl-search! sls3 '(-1 2 3 4))

(apply sl-remove! sls3 (sl-found sls3))

(sl-search! sls3 '(-1 2 3 4))

(null? (sl-found sls3)) ;-> #t

(sl-search! sls3 '(10 1 3))

(apply sl-remove! sls3 (sl-found sls3))

(null? (sl-found sls3)) ;-> #t

;; reorder removing all dups ...

(apply <= (map car
               (skiplist->list
                 (sl-reorder sls3 primary-order secondary-order))))

(<= (sl-count (sl-reorder sls3 primary-order secondary-order))
    (sl-count sls3))

 reorder using only secondary order ...

(apply < (map cadr
              (skiplist->list
                (sl-reorder sls3 secondary-order))))

(>= 10 (sl-count
         (sl-reorder sls3 secondary-order)))

;; restructure ...

(equal? (skiplist->list sls3)
        (skiplist->list (sl-restructure sls3 2 10)))

;; filter ...

((list-of? odd?) (map caddr
                      (skiplist->list
                        (sl-filter sls3 (lambda (x) (odd? (caddr x)))))))

;; map ...

(let ((fn (lambda (x) (* 2 x))))
  (equal?
    (map fn (skiplist->list sls3))
    (skiplist->list (sl-map sls3 fn))))

Requirements

none

Last update

Mar 25, 2019

Author

Juergen Lorenz

Repository

This egg is hosted on the CHICKEN Subversion repository:

https://anonymous@code.call-cc.org/svn/chicken-eggs/release/5/skiplists

If you want to check out the source code repository of this egg and you are not familiar with Subversion, see this page.

License

Copyright (c) 2012-2019, Juergen Lorenz
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
Neither the name of the author nor the names of its contributors may be
used to endorse or promote products derived from this software without
specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

== Version History

; 1.0 : port from chicken-4