Collections · Lesson 17.10

Small vector

Source
Collect the prime factors of numbers in a SmallVector<uint64>, which keeps up to eight of them inside itself without allocating, and watch the longer lists move to the heap.
You'll need: Vector, Move, Out parameter

Many lists are short almost every time: the prime factors of a number, the tags on a post, the children of a tree node, the arguments of a call. A Vector asks its allocator for a block on its first push, so a program that builds a hundred thousand short lists makes a hundred thousand small allocations — each one a trip into the allocator, and each block somewhere else in memory.

A SmallVector<T> avoids most of them. It carries 64 bytes of room inside itself, and its first elements go there. Only when one more element no longer fits do they all move to a block from the allocator:

import Collections::{ CollectionError, SmallVector };

Room inside

How many elements fit inline depends on their size:

Element typeSizeInline capacity
uint81 byte64
int324 bytes16
uint648 bytes8
a struct of 3 float6424 bytes2
a struct over 64 bytes72 bytes0 — a plain vector

InlineCapacity reports it. The program collects prime factors as uint64, so eight fit:

let probe = SmallVector<uint64>(allocator);
PrintLine("inline room: {} uint64 values", probe.InlineCapacity());

Eight is plenty for most numbers. Counted with repetition, the smallest number with nine prime factors is 512, or 2⁹, and a typical number around a billion has about three.

Spilling to the heap

flowchart LR
    s["new small vector"] --> i["elements 1 to 8<br/>inline, no allocation"]
    i -- "9th push" --> h["all moved to a heap block<br/>of 16, then 32, …"]
    h -- "pop, truncate, clear" --> h

The program pushes each factor with an ordinary Push, and IsInline says where they ended up:

let place = factors.IsInline() ? "inline" : "on the heap";
PrintLine("   count {}, {}, capacity {}", factors.Length(), place, factors.Capacity());

9699690, the product of the first eight primes, fills the inline room exactly and still allocates nothing. 1024 has ten factors: the ninth push moves all of them to a heap block of 16, which then grows as a vector's does. Once on the heap, the elements stay there — popping or truncating back to two does not return them inside, as the last line of the output shows.

Everything else is a vector

Apart from where the elements live, a small vector is used as a vector is:

MethodReturnsDoes
Push! CollectionErroradds at the end; may need a block, and so can fail
TryPop(@out)boolmoves the last element out through a pointer
Get(i)T?a copy of element i, none past the end
AsSliceT[..]a view of the elements
AsMutableSlicevar T[..]a writable view
Truncate, Clear—destroy the elements past a length, or all of them
Length, Capacityuintas for a vector
IsEmptyboolwhether it holds nothing

TryPop reports a miss with a bool and hands the element back through an out-parameter:

var last: uint64 = 0;
if factors.TryPop(@last) {
    PrintLine("popped {}, {} left", last, factors.Length());
}

A small vector owns its elements, so it is move-only. Factor builds one and hands it back with return <-factors;.

The price

The room travels with the small vector, so the small vector itself is larger: 104 bytes for a SmallVector<uint64> against 40 for a Vector<uint64>, whether it holds one element or none. And because its elements may live inside it, they move whenever it moves. A view from AsSlice lasts only until the small vector next changes or is moved — a Push that spills to the heap moves every element at once.

Reach for a small vector when lists are many, short and short-lived, and their usual length is known. For one long list, a plain Vector is smaller and simpler.

The program

The whole lesson is one package in the Examples repository. Its comments explain every step.

Src/Main.rux
// Many lists are short almost every time: the prime factors of a number, the tags on a post, the
// children of a node. A `Vector` asks the allocator for a block on its first push, so a program
// that makes thousands of short lists makes thousands of small allocations.
//
// A `SmallVector<T>` carries 64 bytes of room inside itself. Its first elements go there, and
// only an element that no longer fits sends them all to a block from the allocator, where they
// stay. That is 16 `int32` values, or 8 `uint64` values; a type larger than 64 bytes gets no
// inline room at all and behaves as a plain vector.
//
// The price is that the small vector itself is larger, and that its elements move when it moves:
// a view from `AsSlice` lasts only until the vector next changes or is moved.
import Allocator::{ Allocator, SystemAllocator };
import Collections::{ CollectionError, SmallVector };
import Io::{ Print, PrintLine };

// Collects the prime factors of `number`, smallest first, by dividing out each factor in turn.
func Factor(allocator: Allocator, number: uint64) -> SmallVector<uint64> ! CollectionError {
    var factors = SmallVector<uint64>(allocator);
    var rest = number;
    var divisor: uint64 = 2;
    while divisor * divisor <= rest {
        while rest % divisor == 0 {
            factors.Push(divisor)?;
            rest = rest / divisor;
        }
        divisor = divisor + 1;
    }
    if rest > 1 {
        factors.Push(rest)?;
    }
    // A small vector owns its elements and cannot be copied, so it is moved out. A plain
    // `return factors;` is refused:
    //
    //     error: move-only value 'factors' requires an explicit '<-' in return
    //     help: prefix the return value with '<-', as in 'return <-factors'
    return <-factors;
}

func Main() -> ! CollectionError {
    var system = SystemAllocator();
    let allocator: Allocator = system;

    // `InlineCapacity` is how many elements fit in the 64 bytes.
    let probe = SmallVector<uint64>(allocator);
    PrintLine("inline room: {} uint64 values", probe.InlineCapacity());

    let numbers: uint64[5] = [97, 360, 9699690, 1024, 3072000];
    for number in numbers[..] {
        let factors = Factor(allocator, number)?;
        Print("{:>8} =", number);
        for factor in factors.AsSlice() {
            Print(" {}", factor);
        }
        // `IsInline` says whether the elements still live inside the small vector.
        let place = factors.IsInline() ? "inline" : "on the heap";
        PrintLine("   count {}, {}, capacity {}", factors.Length(), place, factors.Capacity());
    }

    // A small vector also pops, truncates and clears like a vector. Once its elements have moved
    // to the heap they stay there, even when only a few are left.
    var factors = Factor(allocator, 1024)?;
    var last: uint64 = 0;
    if factors.TryPop(@last) {
        PrintLine("popped {}, {} left", last, factors.Length());
    }
    factors.Truncate(2);
    PrintLine("after truncating: {} left, inline: {}", factors.Length(), factors.IsInline());
}

Besides Io, its Rux.toml lists Allocator and Collections under [Dependencies].

Run it

cd Examples/Collections/SmallVector
rux run
inline room: 8 uint64 values
      97 = 97   count 1, inline, capacity 8
     360 = 2 2 2 3 3 5   count 6, inline, capacity 8
 9699690 = 2 3 5 7 11 13 17 19   count 8, inline, capacity 8
    1024 = 2 2 2 2 2 2 2 2 2 2   count 10, on the heap, capacity 16
 3072000 = 2 2 2 2 2 2 2 2 2 2 2 2 2 3 5 5 5   count 17, on the heap, capacity 32
popped 2, 9 left
after truncating: 2 left, inline: false

Common mistakes

Try it yourself

  1. Factor 510510, the product of the first seven primes, and 2. Which stays inline with room to spare?
  2. Make a SmallVector<int32> and push 1 to 20, printing IsInline() and Capacity() after each push. At which push does it move?
  3. Print sizeof(SmallVector<int32>) beside sizeof(Vector<int32>). Why is the difference what it is?

Learn more