Part 19: Files
Everything so far has lived in memory and vanished when the program ended. A file is how data outlives the program — and it is also where a program meets a world it does not control: names that are not valid text, disks that fill up, files that someone else deletes or reads half-written. This part tours the Path package, which names files, and the FileSystem package, which reads, writes and manages them, and it handles every error they throw along the way. It ends with Io's memory streams, which run the same reading and writing code with no file at all.
What you will learn
- Why a path is not a string, and how
OsString,PathandPathBuffertake one apart, build one and tidy one. - Opening, writing, reading and closing a file, with every call that can throw an
IoErrormarked withtry— including the read that reaches the end of the file, which arrives as a thrown error. - Making, listing and removing directories, and asking the filesystem what a name refers to.
- Writing fixed-width binary records, and refusing one that the file cut short.
- Gathering small writes in a buffer, and knowing when the bytes really reach the file.
- Replacing a file so that no reader ever sees it half-written, and scratch files that clean up after themselves.
- Reading and writing bytes in memory through the same
ReaderandWritercode as a file, so a format can be tested without the disk.
A file's life
Every lesson here covers a piece of one sequence: turn text into a path, open the file, move bytes, close it — and check each step, because each one asks the operating system for something it may refuse.
sequenceDiagram
participant P as Program
participant F as `File`
participant OS as Operating system
Note over P: `OsString::FromText(text)`<br/>`Path::FromView(…)`
P->>F: `try File::Open(path, Writing())`
F->>OS: create the file, or empty it
P->>F: `try WriteAll(file, bytes)`
F->>OS: `Write`, again and again until every byte is out
P->>F: `try Close()`
OS-->>P: done, or an error delayed until now
P->>F: `try File::Open(path, Reading())`
loop until the end of the file
P->>F: `try Read(buffer[..]) catch { … }`
F-->>P: a count of bytes, or a thrown `IoError`
end
Note over P: `IsEnd()` is `true`: the file is read
P->>F: `try Close()`
P->>OS: `try DeleteFile(path)`| You want to… | Use | Lesson |
|---|---|---|
| take a path apart | Components, FileName, Parent | Path |
| build a path from pieces | Join, PathBuffer::Push | Path join |
| compare or display paths | Normalize | Path normalize |
| read or write a text file | File::Open, WriteAll, Read | File |
| list a directory | ReadDirectory, Next | Directory |
| ask whether a file exists, and what it is | MetadataOf | Metadata |
| store numbers as bytes | WriteUint32, ReadUint32, … | Binary |
| make many small writes cheaply | BufferedWriter | Buffered I/O |
| replace a file without a half-written moment | WriteAtomically, AtomicWrite | Atomic file |
| scratch space that removes itself | TemporaryFile | Temporary file |
| run file code on bytes in memory | ByteWriter, ByteReader | Memory streams |
Lessons
| Lesson | What you will learn | |
|---|---|---|
| 19.1 | Path | split a path into its parts, and see why a path is not a string |
| 19.2 | Path join | build a path from parts |
| 19.3 | Path normalize | tidy a path by removing . and .. |
| 19.4 | File | write text to a file and read it back, handling failure at every step |
| 19.5 | Directory | create, list, and remove directories |
| 19.6 | Metadata | ask a file for its size and kind |
| 19.7 | Binary | read and write fixed-width values and raw bytes |
| 19.8 | Buffered I/O | buffer writes and flush them |
| 19.9 | Atomic file | replace a file's contents all at once or not at all |
| 19.10 | Temporary file | a scratch file that cleans up after itself |
| 19.11 | Memory streams | read and write bytes in memory through the same reader and writer interfaces as a file |
Before you start
This part leans hard on Part 9: Errors — most lessons' Main throws an IoError, every call that can fail is marked with try, and the lessons use Catch, Guard, Nested fallible and Coalesce exit. It also uses Move and Destructor from Part 11, Interface value, Interface parameter and Iterator from Part 12, Allocator from Part 15 and Endian from Part 16. Each lesson's package is in the Examples repository's Files/ folder:
cd Examples/Files/File
rux run
The programs create their files inside their own package's Bin/ folder and delete them before they finish. If a run stops part-way, a leftover file or Bin/scratch directory may need deleting by hand.
After this part
Part 20: Utilities covers time, randomness, hashing and UUIDs — including the Timestamp that Metadata reported. Part 21: Data formats then gives your files a structure, and its checkpoint project, Notes, keeps a to-do list as JSON in a file — saved with WriteAtomically, loaded back, and recovered when the file is missing or damaged.
The Rux Reference describes the language features these packages are built from: Interfaces, the mechanism behind Reader and Writer, and Enumerations, the shape of IoErrorKind and FileKind.
18.7 Scan
Turn a week's sales into running totals and running bests with PartialSum and InclusiveScan, record lengths into offsets with ExclusiveScan, fill a counting sequence with Iota, and find a midpoint that cannot overflow with Midpoint.
19.1 Path
Walk a path's components and named parts, and see why a path is not a string.