This SRFI is currently in ``final'' status.
To see an explanation of
each status that a SRFI can hold, see here.
To provide input on this SRFI, please
mail to
<srfi minus 121 at srfi dot schemers dot org>. See
instructions here to
subscribe to the list. You can access previous messages via
the archive of the mailing list.
This SRFI defines utility procedures that create, transform, and consume generators.
A generator is simply a procedure with no arguments that works
as a source of a series of values. Every time it is called,
it yields a value. Generators may be finite or infinite; a finite
generator returns an end-of-file object to indicate that it is exhausted.
For example, read-char, read-line,
and read are generators that
generate characters, lines, and objects from the current input port.
Generators provide lightweight laziness.
The main purpose of generators is high performance. Although SRFI 41 streams can do everything generators can do and more, SRFI 41 uses lazy pairs that require making a thunk for every item. Generators can generate items without consing, so they are very lightweight and are useful for implementing simple on-demand calculations.
Existing examples of generators are readers from the current input port and SRFI 27 random numbers. If Scheme had streams as one of its built-in abstractions, these would have been naturally represented by lazy streams. But Scheme usually does not expose this kind of API using lazy streams. Generator-like interfaces are common, so it seems worthwhile to have some common idioms extracted into a library.
Calling a generator is a side-effecting construct; you can't safely backtrack, for example, as you can with streams. Persistent lazy sequences based on generators and ordinary Scheme pairs (which are heavier weight than generators, but lighter weight than lazy pairs) is the subject of SRFI 127. Of course the efficiency of streams depends on the implementation. Some implementations may have have super-light thunk creation. But in most, thunk creation is probably slower than simple consing.
The generators of this SRFI don't belong to a disjoint type.
They are just procedures that conform to a calling convention,
so you can construct a generator with lambda. The constructors
of this SRFI are provided for convenience.
Any procedure that can be called with no arguments can serve as a
generator.
Using an end-of-file object to indicate that there is no more input makes it impossible to include such an object in the stream of generated values. However, it is compatible with the existing design of input ports, and it makes for more compact code than returning a user-specified termination object (as in Common Lisp) or returning multiple values. (Note that some generators are infinite in length, and never return an end-of-file object.)
The combination of make-for-each-generator and
generator-unfold makes it possible to convert any
collection that has a for-each procedure into any collection that has
an unfold constructor. This generalizes such conversion procedures as
&list->vector and string->list.
These procedures are drawn from the Gauche core and the Gauche
module gauche.generator, with
some renaming to make them more systematic, and with a few additions
from the Python library
itertools.
Consequently, Shiro Kawai,
the author of Gauche and its specifications, is listed as first author
of this SRFI. John Cowan served as editor and shepherd. Thomas Gilray
provided the sample implementation and a valuable critique of the SRFI.
Special acknowledgements to Kragen Javier Sitaker for his extensive review.
Generators can be divided into two classes, finite and infinite. Both kinds of generators can be invoked an indefinite number of times. After a finite generator has generated all its values, it will return an end-of-file object for all subsequent calls. A generator is said to be exhausted if calling it will return an end-of-file object. By definition, infinite generators can never be exhausted.
A generator is said to be in an undefined state if it
cannot be determined exactly how many values it has generated.
This arises because it is impossible to tell by inspecting a
generator whether it is exhausted or not. For example,
(generator-fold + 0 (generator 1 2 3) (generator 1 2))
will compute 0 + 1 + 1 + 2 + 2 = 6, at which time the second
generator will be exhausted. If the first generator is invoked,
however, it may return either 3 or an end-of-file object,
depending on whether the implementation of generator-fold
has invoked it or not.
Therefore, the first generator is said to be in an undefined state.
The result of a generator constructor is just a procedure,
so printing it doesn't show much. In the examples in this section
we use generator->list to convert the generator to a list.
These procedures have names ending with generator.
generator arg …The simplest finite generator. Generates each of its arguments in turn. When no arguments are provided, it returns an empty generator that generates no values.
make-iota-generator count [ start [ step ] ]Creates a finite generator of a sequence of count numbers. The sequence begins with start (which defaults to 0) and increases by step (which defaults to 1). If both start and step are exact, it generates exact numbers; otherwise it generates inexact numbers. The exactness of count doesn't affect the exactness of the results.
(generator->list (make-iota-generator 3 8)) ⇒ (8 9 10) | |
(generator->list (make-iota-generator 3 8 2)) ⇒ (8 10 12) |
make-range-generator start [ end [ step ] ]Creates a generator of a sequence of numbers. The sequence begins with start, increases by step (default 1), and continues while the number is less than end, or forever if end is omitted. If both start and step are exact, it generates exact numbers; otherwise it generates inexact numbers. The exactness of end doesn't affect the exactness of the results.
(generator->list (make-range-generator 3) 4) ⇒ (3 4 5 6) | |
(generator->list (make-range-generator 3 8)) ⇒ (3 4 5 6 7) | |
(generator->list (make-range-generator 3 8 2)) ⇒ (3 5 7) |
make-coroutine-generator procCreates a generator from a coroutine.
The proc argument is a procedure that takes one argument, yield. When
called, make-coroutine-generator immediately returns
a generator g. When g is called, proc runs
until it calls yield. Calling yield causes
the execution of proc to be suspended, and g returns the value passed
to yield.
Whether this generator is finite or infinite depends on the behavior of proc. If proc returns, it is the end of the sequence — g returns an end-of-file object from then on. The return value of proc is ignored.
The following code creates a generator that produces a series
0, 1, and 2 (effectively the same as (make-range-generator 0 3) and binds
it to g.
(define g
(make-coroutine-generator
(lambda (yield) (let loop ((i 0))
(when (< i 3) (yield i) (loop (+ i 1)))))))
(generator->list g) ⇒ (0 1 2)
|
list->generator lisvector->generator vec [ start [ end ] ]reverse-vector->generator vec [ start [ end ] ]string->generator str [ start [ end ] ]bytevector->generator bytevector [ start [ end ] ]These procedures return generators that yield each element of the given argument. Mutating the underlying object will affect the results of the generator.
(generator->list (list->generator '(1 2 3 4 5))) ⇒ (1 2 3 4 5) (generator->list (vector->generator '#(1 2 3 4 5))) ⇒ (1 2 3 4 5) (generator->list (reverse-vector->generator '#(1 2 3 4 5))) ⇒ (5 4 3 2 1) (generator->list (string->generator "abcde")) ⇒ (#\a #\b #\c #\d #\e) |
The generators returned by the constructors are exhausted once all elements are retrieved; the optional start-th and end-th arguments can limit the range the generator walks across.
For reverse-vector->generator, the first value is the element right before
the end-th element, and the last value is the start-th
element.
For all the other constructors, the first value the generator yields
is the start-th element, and it ends right before the end-th element.
(generator->list (vector->generator '#(a b c d e) 2)) ⇒ (c d e) (generator->list (vector->generator '#(a b c d e) 2 4)) ⇒ (c d) (generator->list (reverse-vector->generator '#(a b c d e) 2)) ⇒ (e d c) (generator->list (reverse-vector->generator '#(a b c d e) 2 4)) ⇒ (d c) (generator->list (reverse-vector->generator '#(a b c d e) 0 2)) ⇒ (b a) |
make-for-each-generator for-each objA generator constructor that converts any collection obj to
a generator that returns its elements using a
for-each procedure appropriate for obj. This must
be a procedure that when called as (for-each proc obj) calls
proc on each element of obj. Examples of such
procedures are for-each, string-for-each,
and vector-for-each from R7RS. The value returned by
for-each is ignored.
The generator is finite if the collection is finite, which would
typically be the case.
The collections need not be conventional ones (lists, strings, etc.)
as long as for-each can invoke a procedure on everything that
counts as a member. For example,
the following procedure allows for-each-generator to generate
the digits of an integer from least to most significant:
(define (for-each-digit proc n)
(when (> n 0)
(let-values (((div rem) (truncate/ n 10)))
(proc rem)
(for-each-digit proc div))))
|
make-unfold-generator stop? mapper successor seedA generator constructor similar to
SRFI 1's unfold.
The stop? predicate takes a seed value and determines whether to stop. The mapper procedure calculates a value to be returned by the generator from a seed value. The successor procedure calculates the next seed value from the current seed value.
For each call of the resulting generator, stop? is called with the current seed value. If it returns true, then the generator returns an end-of-file object. Otherwise, it applies mapper to the current seed value to get the value to return, and uses successor to update the seed value.
This generator is finite unless stop? never returns true.
(generator->list (make-unfold-generator
(lambda (s) (> s 5))
(lambda (s) (* s 2))
(lambda (s) (+ s 1))
0))
⇒ (0 2 4 6 8 10)
|
The following procedures accept one or more generators and return a new generator without consuming any elements from the source generator(s). In general, the result will be a finite generator if the arguments are.
The names of these procedures are prefixed with g.
gcons* item … genReturns a generator that adds items in front of gen. Once the items have been consumed, the generator is guaranteed to tail-call gen.
(generator->list (gcons* 'a 'b (make-range-generator 0 2))) ⇒ (a b 0 1) |
gappend gen …Returns a generator that yields the items from the first given generator, and once it is exhausted, from the second generator, and so on.
(generator->list (gappend (make-range-generator 0 3) (make-range-generator 0 2))) ⇒ (0 1 2 0 1) (generator->list (gappend)) ⇒ () |
gcombine proc seed gen gen2 …A generator for mapping with state. It yields a sequence of sub-folds over proc.
The proc argument is a procedure that takes as many arguments
as the input generators plus one. It is called as
(proc v1 v2 … seed),
where v1, v2, … are
the values yielded from the input generators, and seed is the
current seed value. It must return two values, the yielding value
and the next seed.
The result generator is exhausted when any of the genn
generators is exhausted, at which time all the others are in an undefined state.
gfilter pred gengremove pred genReturn generators that yield the items from the source generator, except those on which pred answers false or true respectively.
gtake gen k [ padding ]gdrop gen kThese are generator analogues of SRFI 1
take and drop. Gtake returns a generator
that yields (at most) the first k items
of the source generator, while gdrop returns a generator
that skips the first k items of the source generator.
These won't complain if the source generator is exhausted before generating
k items. By default, the generator returned by gtake
terminates when the source generator does, but if you provide the padding argument,
then the returned generator will yield exactly k items, using the padding value
as needed to provide sufficient additional values.
gtake-while pred gengdrop-while pred genThe generator analogues of SRFI-1 take-while and drop-while. The generator returned
from gtake-while yields items from the source generator
as long as pred returns true for each. The generator returned
from gdrop-while first reads and discards values from the source generator
while pred returns true for them, then starts yielding items returned by the source.
gdelete item gen [ = ]Creates a generator that returns whatever gen returns, except for any items that
are the same as item in the sense of =, which defaults to equal?.
The = predicate is passed exactly two arguments, of which the first was generated
by gen before the second.
(generator->list (gdelete 3 (generator 1 2 3 4 5 3 6 7))) ⇒ (1 2 4 5 6 7) |
gdelete-neighbor-dups gen [ = ]Creates a generator that returns whatever gen returns, except for any items
that are equal to the preceding item in the sense of =, which defaults to equal?.
The = predicate is passed exactly two arguments, of which the first was generated
by gen before the second.
(generator->list (gdelete-neighbor-dups (list->generator '(a a b c a a a d c)))) ⇒ (a b c a d c) |
gindex value-gen index-genCreates a generator that returns elements of value-gen specified by the indices (non-negative exact integers) generated by index-gen. It is an error if the indices are not strictly increasing, or if any index exceeds the number of elements generated by value-gen. The result generator is exhausted when either generator is exhausted, at which time the other is in an undefined state.
(generator->list (gindex (list->generator '(a b c d e f))
(list->generator '(0 2 4))))
⇒ (a c e)
|
gselect value-gen truth-genCreates a generator that returns elements of value-gen that correspond to the values generated by truth-gen. If the current value of truth-gen is true, the current value of value-gen is generated, but otherwise not. The result generator is exhausted when either generator is exhausted, at which time the other is in an undefined state.
(generator->list (gselect (list->generator '(a b c d e f))
(list->generator '(#t #f #f #t #t #f))))
⇒ (a d e)
|
Unless otherwise noted,
these procedures consume all the values available from the generator that is passed to them,
and therefore will not return if one or more generator arguments are infinite.
They have names prefixed with generator.
generator->list generator [ k ]Reads items from generator and returns a newly allocated list of them. By default, it reads until the generator is exhausted.
If an optional argument k is given, it must be a non-negative integer, and the list ends when either k items are consumed, or generator is exhausted; therefore generator can be infinite in this case.
generator->reverse-list generator [ k ]Reads items from generator and returns a newly allocated list of them in reverse order. By default, this reads until the generator is exhausted.
If an optional argument k is given, it must be a non-negative integer, and the list ends when either k items are read, or generator is exhausted; therefore generator can be infinite in this case.
generator->vector generator [ k ]Reads items from generator and returns a newly allocated vector of them. By default, it reads until the generator is exhausted.
If an optional argument k is given, it must be a non-negative integer, and the list ends when either k items are consumed, or generator is exhausted; therefore generator can be infinite in this case.
generator->vector! vector at generatorReads items from generator and puts them into vector starting at index at, until vector is full or generator is exhausted. Generator can be infinite. The number of elements generated is returned.
generator->string generator [ k ]Reads items from generator and returns a newly allocated string of them. It is an error if the items are not characters. By default, it reads until the generator is exhausted.
If an optional argument k is given, it must be a non-negative integer, and the string ends when either k items are consumed, or generator is exhausted; therefore generator can be infinite in this case.
generator-fold proc seed gen1 gen2 …Works like SRFI 1 fold on the values generated by the generator
arguments.
When one generator is given, for each value v generated
by gen,
proc is called as (proc v r), where
r is the current accumulated result; the initial value of the
accumulated result is seed,
and the return value from proc becomes the next accumulated result.
When gen is exhausted, the accumulated result at that time is returned
from generator-fold.
When more than one generator is given, proc is invoked on the values returned by all the generator arguments followed by the current accumulated result. The procedure terminates when any of the genn generators is exhausted, at which time all the others are in an undefined state.
(with-input-from-string "a b c d e" (lambda () (generator-fold cons 'z read))) ⇒ (e d c b a . z) |
generator-for-each proc gen gen2 …A generator analogue of for-each that consumes generated values using side effects.
Repeatedly applies proc on
the values yielded by gen, gen2 … until any one of
the generators is exhausted. The values returned from proc are discarded.
The procedure terminates when any of the genn
generators is exhausted, at which time all the others are in an undefined state.
Returns an unspecified value.
generator-find pred genReturns the first item from the generator gen that satisfies
the predicate pred, or #f if no such item is found before
gen is exhausted.
If gen is infinite, generator-find will not return
if it cannot find an appropriate item.
generator-count pred genReturns the number of items available from the generator gen that satisfy the predicate pred.
generator-any pred genApplies pred to each item from gen. As soon
as it yields a true value, the value is returned without consuming the
rest of gen.
If gen is exhausted, returns #f.
generator-every pred genApplies pred to each item from gen. As soon
as it yields a false value, the value is returned without consuming the
rest of gen.
If gen is exhausted, returns the last value returned by
pred, or #t if pred was never called.
generator-unfold gen unfold arg ...
Equivalent to (unfold eof-object? (lambda (x) x)
(lambda (x) (gen)) (gen) arg ...).
The values of gen
are unfolded into the collection that unfold creates.
The signature of the unfold procedure is (unfold stop? mapper successor seed args ...).
Note that the
vector-unfold and vector-unfold-right
of SRFI 43
and SRFI 133
do not have this signature and cannot be used with this procedure.
To unfold into a vector, use SRFI 1's unfold and then apply list->vector
to the result.
;; Iterates over string and unfolds into a list using SRFI 1 unfold (generator-unfold (make-for-each-generator string-for-each "abc") unfold) ⇒ (#\a #\b #\c) |
The sample implementation is in the SRFI 121 repository. It contains the following files:
generators-impl.scm - implementation of generatorsr7rs-shim.scm - supplementary code for non-R7RS systemsgenerators.sld - R7RS librarygenerators.scm - Chicken librarygenerators-test.scm - Chicken test-egg test filePermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.