Atomic file

Overwriting a file in place has a dangerous moment. Opening it with OpenOptions::Writing() empties it at once, and until the last byte is written the file holds only part of the new contents. A crash, a full disk, or a reader arriving at the wrong time finds a broken file — and the old one is already gone. Settings files, saved games and databases all need a way round that moment, and the FileSystem package has one.

Write elsewhere, then rename

func WriteAtomically(allocator, target: Path, contents: char8[..]) -> ! IoError

WriteAtomically never touches the target until the new contents are complete. It writes them to a new temporary file in the same directory as the target, forces them to the disk, and then renames the temporary file onto the target's name in one step the filesystem cannot interrupt:

flowchart LR
    b["create a temporary file<br/>beside the target"] --> w["write the new<br/>contents into it"]
    w --> s["sync it<br/>to the disk"]
    s --> r["rename it onto<br/>the target's name"]
    r --> n["from here on, readers<br/>see the new contents"]

Until the last step the target is never opened, so anyone reading it meanwhile finds the complete old contents.

The same directory matters: a rename is only atomic within one volume, and the system's temporary folder is often on another, where a "rename" silently becomes a copy and a delete.

For a file whose whole contents are in hand, one call does it:

WriteAtomically(allocator, path, "volume = 3")?;

The same steps by hand

WriteAtomically is built on AtomicWrite, and the second half of the program uses it directly to look inside. Begin creates the temporary file; the value is a Writer, so WriteAll writes into it:

var replacement = AtomicWrite::Begin(allocator, path)?;
WriteAll(replacement, "volume = 11")?;
Show(allocator, "while writing", path)?;

At this point the new contents exist — in the temporary file. Show opens the target and still finds volume = 3. Only Commit makes the switch:

replacement.Commit(allocator)?;
Show(allocator, "after commit", path)?;

Commit syncs the temporary file and renames it onto the target, and from then on the target reads volume = 11.

A replacement you do not commit

If you change your mind, Abandon deletes the temporary file and leaves the target alone. Simply dropping the value does the same: its destructor throws the partial file away. So an early return, or a ? that passes a failure on part-way through writing, leaves the old contents exactly where they were — which is the behaviour that makes the type worth using.

What it does not promise

Atomic is not the same as durable. After Commit, the new contents are on the disk, but the rename itself — the directory entry — may not be: making that survive a power cut needs the directory synced too, which Windows does not offer. So after a crash at the wrong moment the target may still hold the old version. Old or new, but always whole.

The Show helper reads the whole file into one buffer. Its 64 bytes are plenty for these settings; a file of unknown size would need a reading loop like the one in File.

The program

The whole lesson is one package in the Examples repository. Its comments explain every step.

// Overwriting a file in place has a dangerous moment. Opening it with `Writing` empties it, and
// until the last byte is written the file holds part of the new contents. A crash, a full disk
// or a reader arriving at the wrong time sees a broken file, and the old one is already gone.
//
// `FileSystem::WriteAtomically` avoids that moment:
//
//     func WriteAtomically(allocator, target: Path, contents: char8[..]) -> ! IoError
//
// It writes the contents to a new temporary file in the same directory as the target, forces
// them to the disk, and then renames the temporary file onto the target's name in one step. The
// same directory matters, because a rename is only atomic within one volume.
//
// What it promises: anyone opening the target at any moment finds the complete old contents or
// the complete new ones, never a mix, never an empty file and never a missing one. If it fails,
// the target still holds the old contents, and the temporary file is deleted.
//
// What it does not promise: that the rename itself survives a power cut. That would need the
// directory synced to disk too, which Windows does not offer, so after a crash at the wrong
// moment the target may still hold the old version. Old or new, but always whole.
//
// The work is done by `AtomicWrite`, which the second half uses directly to look inside.
import Allocator::{ Allocator, SystemAllocator };
import FileSystem::{ AtomicWrite, DeleteFile, File, OpenOptions, WriteAtomically };
import Io::{ IoError, PrintLine, ReadExact, WriteAll };
import Path::{ OsString, Path };
import Text::TextError;

// Prints what the file holds right now. The files here are short, so one buffer is enough.
func Show(allocator: Allocator, label: char8[..], path: Path) -> ! IoError {
    var file = File::Open(allocator, path, OpenOptions::Reading())?;
    let size = file.Size()? as uint;
    var buffer: char8[64];
    ReadExact(file, buffer[..size])?;
    file.Close()?;
    PrintLine("{:16} {}", label, buffer[..size]);
}

func Main() -> ! IoError | TextError {
    var system = SystemAllocator();
    let allocator: Allocator = system;
    var holder = OsString::FromText(allocator, "Bin/settings.txt")?;
    let path = Path::FromView(holder.View());

    WriteAtomically(allocator, path, "volume = 3")?;
    Show(allocator, "first write", path)?;

    // The same steps by hand. While the new contents are being written they live in the
    // temporary file, and the target still holds the old ones.
    var replacement = AtomicWrite::Begin(allocator, path)?;
    WriteAll(replacement, "volume = 11")?;
    Show(allocator, "while writing", path)?;

    // `Commit` syncs the temporary file and renames it onto the target.
    replacement.Commit(allocator)?;
    Show(allocator, "after commit", path)?;

    DeleteFile(allocator, path)?;
}

Besides Io, its Rux.toml lists Allocator, FileSystem, Path and Text under [Dependencies].

Run it

cd Examples/Files/AtomicFile
rux run

first write      volume = 3
while writing    volume = 3
after commit     volume = 11

Common mistakes

Try it yourself

1. Replace Commit with replacement.Abandon(allocator)?; and check that the target still says volume = 3.
2. Write a volume = 11 setting in place instead, with File::Open and OpenOptions::Writing(), and call Show between opening and writing. What does a reader see at that moment?
3. Use AtomicWrite to write the contents in three WriteAll calls — volume, =, 11 — and commit once.
4. Call Show on the target from a second Show call placed after Begin but before WriteAll. Is the target empty, as it would be with Writing?

Learn more

• File — opening, writing and closing, and why Close is fallible
• Temporary file — the scratch file AtomicWrite is built on
• Destructor — how dropping an uncommitted replacement cleans up
• Binary — the cut-short record that writing atomically prevents