Algorithms · Lesson 18.6

Iterator adaptors

Source
Chain Core's lazy adaptors — Map, Filter, Skip, Take, Enumerate, Zip and Chain — over slices and over an endless iterator, and see that each item is computed only when it is asked for.
You'll need: Iterator, Iterable, Callback

The Iterator lesson built a type whose Next hands out one item at a time, and for walked it. Very often the walk you want is a small change to one you already have: the same scores, curved; only the passes; the second page of results; each item with its position. Writing a new iterator type for every such change would be tedious.

An adaptor is an iterator that wraps another one and changes what comes out. Core provides seven, and because each adaptor is an iterator too, they nest:

import Core::{ Chain, Enumerate, Filter, Indexed, Map, Skip, Take, Walk, Zip };

Cursors and Walk

An adaptor takes anything whose Next returns T?. Core calls that shape a Cursor<T>, and these all have it:

  • a container's iterator, such as a vector's Iterate();
  • an iterator type of your own, like the program's Counter;
  • another adaptor.

A slice does not: for walks a slice directly, but it has no Next. Walk wraps one in a cursor, so a chain over a slice starts there:

for score in Map(Walk(scores[..]), Curve) {
    Print(" {}", score);
}

The seven adaptors

AdaptorHands outItem type
Map(c, f)f(item) for each itemwhat f returns
Filter(c, keep)the items for which keep(item) is truesame as c
Take(c, n)the first n items, then stopssame as c
Skip(c, n)everything after the first n itemssame as c
Enumerate(c)each item with its position, counting from 0Indexed<T>
Zip(a, b)an item from each, side by side, until either runs outPair<A, B>
Chain(a, b)every item of a, then every item of bsame as both

Map and Filter take a named function, as a callback. Indexed<T> has the fields index and item; Pair<A, B> has first and second.

No type arguments are written anywhere. Each adaptor works its item type out from the cursor it wraps, and from the function's return type for Map.

Reading a chain

A chain reads from the inside out. The innermost call is the source; each call around it is one more step:

for score in Filter(Map(Walk(scores[..]), Curve), Passed) {
    Print(" {}", score);
}

Walk the scores, curve each, keep the passes. The order matters: filtering before curving would test the raw scores, and fail the 45 and 49 that the curve lifts to 50 and 54. In the same way, Enumerate before Filter keeps the original positions — the program reports the failures at 1, 3 and 5 — where Enumerate after Filter would number the survivors 0, 1, 2.

Lazy

Building a chain computes nothing. When for asks the outermost adaptor for an item, it asks the one inside it, and so on down to the source, one item at a time:

flowchart TD
    f["`for` wants an item"] --> t["`Take`: fewer than 3 handed out?<br/>Then ask `Filter`"]
    t --> fi["`Filter`: ask `Map` again<br/>until an item is even"]
    fi --> m["`Map`: ask `Counter`,<br/>square the answer"]
    m --> c["`Counter`: hand out<br/>the next number"]

So no list of intermediate results is ever built, and a chain can draw on a source that never ends. The program's Counter counts up for ever, and prints each time it is asked:

for value in Take(Filter(Map(Counter { next: 1 }, Square), IsEven), 3) {
    Print(" -> {}", value);
}

The output shows each number asked for just before the result it leads to, and the asking stops at 6: after the third even square, Take stops asking.

The program

The whole lesson is one package in the Examples repository. Its comments explain every step.

Src/Main.rux
// An iterator hands out one item each time its `Next` is called. An adaptor is an iterator that
// wraps another one and changes what comes out: `Map` passes each item through a function,
// `Filter` drops the items that fail a test, `Take` stops after a number of them, and so on.
// Core provides seven, and because each is an iterator itself they nest, each one wrapping the
// next:
//
//     Take(Filter(Map(source, Square), IsEven), 3)
//
// Read that from the inside out: square each item, keep the even squares, stop after three.
//
// Adaptors are lazy. Building the chain computes nothing; each adaptor asks the one inside it
// for an item only when it is itself asked for one. So no list of results is ever built, and a
// chain can even draw from a source that never ends, as long as something stops asking.
//
// An adaptor takes anything whose `Next` returns `T?` — Core calls that a `Cursor<T>`. That is
// a container's iterator, another adaptor, or a slice turned into one by `Walk`.
import Core::{ Chain, Enumerate, Filter, Indexed, Map, Skip, Take, Walk, Zip };
import Io::{ Print, PrintLine };

// Counts up from a number, for ever, and says each time it is asked.
struct Counter {
    next: int32;
}

extend Counter {
    func Next(self: &var Counter) -> int32? {
        Print(" (asked: {})", self.next);
        self.next = self.next + 1;
        return self.next - 1;
    }
}

func Curve(score: int32) -> int32 {
    return score + 5;
}

func Passed(score: int32) -> bool {
    return score >= 50;
}

func Failed(entry: Indexed<int32>) -> bool {
    return entry.item < 50;
}

func Square(value: int32) -> int32 {
    return value * value;
}

func IsEven(value: int32) -> bool {
    return value % 2 == 0;
}

func Main() -> int {
    let names: char8[..][6] = ["ada", "ben", "cai", "dee", "eve", "fay"];
    let scores: int32[6] = [72, 45, 90, 38, 66, 49];

    // A slice is not an iterator; `Walk` makes one. `Map` then hands out each score curved.
    // Leave out the `Walk` and the slice fails the bound:
    //
    //     for score in Map(scores[..], Curve) { ... }
    //         error: type argument 'int32[..]' does not satisfy interface bound 'Cursor' on type
    //                parameter 'I'
    Print("curved:   ");
    for score in Map(Walk(scores[..]), Curve) {
        Print(" {}", score);
    }
    PrintLine();

    // Adaptors nest: curve every score, then keep the passes.
    Print("passed:   ");
    for score in Filter(Map(Walk(scores[..]), Curve), Passed) {
        Print(" {}", score);
    }
    PrintLine();

    // `Skip` and `Take` cut out a window: the second page of two.
    Print("page 2:   ");
    for score in Take(Skip(Walk(scores[..]), 2), 2) {
        Print(" {}", score);
    }
    PrintLine();

    // `Enumerate` pairs each item with its position, as an `Indexed<int32>`, so a filter after
    // it still knows where each item stood.
    Print("failed at:");
    for entry in Filter(Enumerate(Walk(scores[..])), Failed) {
        Print(" {}", entry.index);
    }
    PrintLine();

    // `Zip` walks two sources side by side, as a `Pair`, and stops when either runs out.
    for pair in Take(Zip(Walk(names[..]), Walk(scores[..])), 3) {
        PrintLine("{}  {}", pair.first, pair.second);
    }

    // `Chain` reads one source to its end, then the other.
    let late: int32[2] = [81, 57];
    Print("all:      ");
    for score in Chain(Walk(scores[..]), Walk(late[..])) {
        Print(" {}", score);
    }
    PrintLine();

    // Laziness: the counter never ends, yet it is asked for only six numbers, one at a time as
    // each result is wanted, because `Take` stops asking once it has handed out three.
    Print("lazy:     ");
    for value in Take(Filter(Map(Counter { next: 1 }, Square), IsEven), 3) {
        Print(" -> {}", value);
    }
    PrintLine();
    return 0;
}

Besides Io, its Rux.toml lists Core under [Dependencies].

Run it

cd Examples/Algorithms/IteratorAdaptor
rux run
curved:    77 50 95 43 71 54
passed:    77 50 95 71 54
page 2:    90 38
failed at: 1 3 5
ada  72
ben  45
cai  90
all:       72 45 90 38 66 49 81 57
lazy:      (asked: 1) (asked: 2) -> 4 (asked: 3) (asked: 4) -> 16 (asked: 5) (asked: 6) -> 36

Common mistakes

Try it yourself

  1. Print the names of the students who passed after the curve: Zip the names with the curved scores, then Filter the pairs.
  2. Print the scores with a rank, starting from 1, using Enumerate and entry.index + 1.
  3. Chain scores and late, Skip the first five and Take two. Predict the output before you run it.
  4. Write a cursor Repeat that hands out the same value for ever, and Zip it with the scores.

Learn more