Cover

About this book

In this book, we will learn how Linux works and dive deep into the kernel’s machinery with hands-on examples, both in Rust and Go. We will first build intuition by reasoning and quickly prototype the data structures behind each kernel mechanism. Then we will run the same experiments against the real kernel and compare the results.

Linux runs most of the world’s servers and phones, and the machines that train and serve AI models. Containers, Kubernetes, and cloud platforms are all built on a small set of kernel mechanisms: pipes, processes, file descriptors, signals, virtual memory, namespaces, and control groups. When we understand these mechanisms, we can reason about what a container is, why a service stalls, and where a machine’s memory has gone.

The kernel is tens of millions of lines of C, and knowing all of it can look like a pipe dream. The ideas inside it are much smaller than the code. A pipe is a ring of pages indexed by a pair of counters. A container is a process with a restricted view of the system. Each of these is small enough for us to build ourselves, so we will build them one at a time, starting with the pipe.

The examples run on any Linux machine. The output shown in this book was recorded on a Jetson Orin NX, an arm64 machine running Linux 5.15.

About the cover

The cover shows a penguin asleep in the snow. Its dream is visible through the open end of a steel pipe: a sunny hill, a desktop computer that runs Linux, and the years 1999, 2009 and 2019 struck out. Beside the sleeper a dumpster labelled teraSlop is on fire, and a crab toasts a marshmallow over it.

Each chapter opens with a different animal at work on the chapter’s subject.

The gopher and the crab appear because the programs of the book are written in Go and in Rust. The plates are drawn by tools/art.py.

Pipes

A pipe is a one-way stream of bytes between processes, buffered in a fixed amount of kernel memory. The shell creates one for each | on a command line. In seq 1 100000 | grep 7 | wc -l, the first pipe carries the output of seq to the input of grep, and the second carries the output of grep to the input of wc.

In this chapter, we will specify what a pipe has to do, measure how the Linux pipe behaves, and then implement the same design ourselves. We will write pipefacts, which measures a kernel pipe, handpipe, which implements a pipe in user space with the kernel’s data structure, and pipeline, which connects real commands with kernel pipes the way a shell does. Each program is written in Go and in Rust.

Why a pipe

Consider the job of counting how many of the numbers from 1 to 100000 contain the digit 7. seq 1 100000 prints the numbers, 588895 bytes of text, and grep 7 | wc -l counts the matching lines. The output of the first program has to reach the second somehow.

The obvious way is a temporary file: run seq and store its output, then run grep on the file. This works, and it has 3 costs. The whole output is stored, although each number is needed only once. The second program cannot start until the first has finished. And if the output were endless, as it is for a program that follows a log, the method would not work at all.

A pipe removes all 3. It is a small buffer in memory with a writer on one side and a reader on the other. The writer adds bytes, the reader removes them, and both programs run at the same time. The buffer holds 65536 bytes at most, whether 600 kilobytes pass through it or 600 gigabytes.

A small buffer between 2 running programs raises questions that a file does not. What happens when the buffer is full? What happens when it is empty? How does the reader learn that the writer has finished? The answers are the behaviour of a pipe, and the next section lists them as requirements.

Requirements

Linux runs each program as a process, which has its own memory. A process cannot read or write the memory of another. The kernel is the part of the operating system that manages memory and devices for all processes, and a process asks it for a service with a system call: a function call that enters the kernel and returns a result. read and write are system calls. Each takes a file descriptor, a small integer that names an open file or another source of bytes.

Since processes share no memory, a stream of bytes between them has to pass through the kernel. The table lists what such a stream must provide and the mechanism a pipe uses to provide it.

A call blocks when the kernel suspends the calling thread until the call can proceed. A suspended thread consumes no processor time.

The interface

The pipe2 system call creates a pipe and returns a pair of file descriptors: the read end, then the write end. Bytes passed to write on the write end are appended to the pipe’s buffer, and read on the read end removes them from the front.

A pipe has no name in the file system and stores nothing on disk. It exists for as long as some process holds a descriptor for either end. Linux lists the descriptors of a process under /proc/<pid>/fd, where <pid> is the process number, so we can observe a pipe from outside. For sleep 47 | sleep 48, with the processes numbered 4121903 and 4121904, descriptor 1 (standard output) of the first and descriptor 0 (standard input) of the second refer to the same pipe object:

$ ls -l /proc/4121903/fd/1 /proc/4121904/fd/0
/proc/4121903/fd/1 -> pipe:[33835818]
/proc/4121904/fd/0 -> pipe:[33835818]

The listing is read as follows.

The same can be seen from inside a pipeline. Here ls lists its own descriptors while its output goes to cat. The lines for descriptors 1 and 3 are shown: descriptor 1 of ls is a pipe, and descriptor 3 is the directory it is reading.

$ ls -l /proc/self/fd | cat
1 -> pipe:[35453230]
3 -> /proc/559492/fd

A pipe carries bytes in 1 direction. A program that needs to send and receive creates 2 pipes, or uses a socket (chapter 10). The descriptors, the table they live in and what a child process inherits are the subject of chapter 2. For this chapter it is enough that a descriptor is a number that names 1 end of the pipe.

One write, step by step

Before looking at the data structures, we follow a single call. A process holds the write end of a pipe as descriptor 4 and calls write(4, "hello", 5).

1. The kernel takes entry 4 of the process’s descriptor table and finds a struct file, its record of an open file. The record contains a table of functions, f_op, and for a pipe the write function in that table is pipe_write.
2. pipe_write locks the pipe. A mutex is a lock that 1 thread can hold at a time, and a thread that asks for a held mutex sleeps until it is released. All further steps happen with the lock held, so a reader running at the same moment sees the pipe either before the write or after it.
3. It checks that a read end is open. If none is, it sends the process the signal SIGPIPE and returns the error EPIPE.
4. It looks for room. The bytes can go into the unused end of the page written last, or into a new page if a slot of the buffer is free. If there is no room, the process is put to sleep until a reader makes some, and the step is repeated.
5. It copies the 5 bytes from the memory of the process into the page.
6. It wakes any reader that is asleep waiting for data, releases the lock, and returns 5.

Each step corresponds to a field of the structure that represents a pipe in the kernel.

The kernel’s structures

A pipe is a struct pipe_inode_info, and each slot of its buffer is a struct pipe_buffer. These are the definitions in include/linux/pipe_fs_i.h of the kernel that runs on our machine, Linux 5.15, with the fields this chapter does not use left out.

struct pipe_buffer {
        struct page *page;             /* the page that holds the bytes */
        unsigned int offset, len;      /* where the unread bytes start, and how many */
        unsigned int flags;            /* PIPE_BUF_FLAG_CAN_MERGE: may be appended to */
        /* ... */
};

struct pipe_inode_info {
        struct mutex mutex;                    /* the lock of step 2 */
        wait_queue_head_t rd_wait, wr_wait;    /* sleeping readers, sleeping writers */
        unsigned int head;                     /* slots ever filled */
        unsigned int tail;                     /* slots ever emptied */
        unsigned int max_usage;                /* slots that may be in use */
        unsigned int ring_size;                /* slots in bufs, a power of 2 */
        unsigned int readers;                  /* open read ends */
        unsigned int writers;                  /* open write ends */
        struct pipe_buffer *bufs;              /* the array of slots */
        /* ... */
};

#define PIPE_DEF_BUFFERS        16

The capacity follows from these numbers: 16 slots of 4096 bytes are 65536 bytes. The first measurement below checks it.

Setting up the project

The Go programs share a module, and the Rust programs share a Cargo workspace. The Rust crates use the 2024 edition and Rust 1.99, the current stable release. A rust-toolchain.toml file in the rust directory selects it.

pipefacts calls the kernel through golang.org/x/sys/unix in Go and through the nix crate in Rust. handpipe uses the parking_lot crate in Rust for its lock and condition variables. pipeline needs only the standard library in both languages.

To create the Go module with its dependency, we run:

$ mkdir go && cd go
$ go mod init democratizing-linux
$ go get golang.org/x/sys/unix

This produces the following go.mod. Each program is a directory with a main.go, such as pipefacts/main.go.

module democratizing-linux

go 1.26.0

require golang.org/x/sys v0.49.0

For Rust, we create a directory rust with this workspace manifest in Cargo.toml:

[workspace]
members = []
resolver = "3"

and this rust-toolchain.toml beside it:

[toolchain]
channel = "1.99.0"
components = ["rustfmt", "clippy"]

Then we create the crates and add their dependencies. cargo new adds each crate to members.

$ cargo new pipefacts
$ cargo add -p pipefacts nix --features fs
$ cargo new handpipe
$ cargo add -p handpipe parking_lot
$ cargo new pipeline

The manifest of pipefacts is now:

[package]
name = "pipefacts"
version = "0.1.0"
edition = "2024"
rust-version = "1.99"
publish = false

[dependencies]
nix = { version = "0.31.3", features = ["fs"] }

and the manifest of handpipe:

[package]
name = "handpipe"
version = "0.1.0"
edition = "2024"
rust-version = "1.99"
publish = false

[dependencies]
parking_lot = "0.12.5"

We run a program with go run ./pipefacts from the go directory and with cargo run -p pipefacts from the rust directory.

Measuring a kernel pipe

pipefacts measures the behaviours in the requirements table on a kernel pipe. It creates each pipe with the O_NONBLOCK flag. In this mode, a call that would block returns the error EAGAIN immediately, which lets a thread detect a full or an empty buffer without being suspended. We will come back to O_CLOEXEC when we start other programs.

Listing 1.1 The imports of pipefacts, in Go. code/go/pipefacts/main.go

import (
	"errors"
	"fmt"
	"log"

	"golang.org/x/sys/unix"
)

Listing 1.2 The imports of pipefacts, in Rust. code/rust/pipefacts/src/main.rs

use std::os::fd::OwnedFd;

use nix::errno::Errno;
use nix::fcntl::{FcntlArg, OFlag, fcntl};
use nix::unistd::{pipe2, read, write};

Listing 1.3 Creating a non-blocking pipe, in Go. code/go/pipefacts/main.go

// block is the size of every write the program attempts: one page.
const block = 4096

// newPipe creates a pipe in non-blocking mode: a read or a write that cannot
// proceed returns EAGAIN where it would otherwise suspend the caller.
func newPipe() (reader, writer int, err error) {
	var fds [2]int

	if err := unix.Pipe2(fds[:], unix.O_NONBLOCK|unix.O_CLOEXEC); err != nil {
		return 0, 0, fmt.Errorf("pipe2: %w", err)
	}

	return fds[0], fds[1], nil
}

Listing 1.4 Creating a non-blocking pipe, in Rust. code/rust/pipefacts/src/main.rs

/// The size of every write the program attempts: one page.
const BLOCK: usize = 4096;

/// Creates a pipe in non-blocking mode: a read or a write that cannot proceed
/// returns `EAGAIN` where it would otherwise suspend the caller. The read end
/// is first.
fn new_pipe() -> nix::Result<(OwnedFd, OwnedFd)> {
    pipe2(OFlag::O_NONBLOCK | OFlag::O_CLOEXEC)
}

fill measures free space. It writes 4096-byte blocks until a write returns EAGAIN and returns the total number of bytes accepted. drain removes bytes.

Listing 1.5 Measuring free space by writing until the kernel refuses, in Go. code/go/pipefacts/main.go

// fill writes whole blocks until the kernel refuses one and returns the
// number of bytes accepted.
func fill(writer int) (int, error) {
	data := make([]byte, block)
	total := 0

	for {
		n, err := unix.Write(writer, data)

		if errors.Is(err, unix.EAGAIN) {
			return total, nil
		}

		if err != nil {
			return total, fmt.Errorf("write: %w", err)
		}

		total += n
	}
}

// drain reads and discards up to n bytes.
func drain(reader, n int) error {
	_, err := unix.Read(reader, make([]byte, n))

	return err
}

Listing 1.6 Measuring free space by writing until the kernel refuses, in Rust. code/rust/pipefacts/src/main.rs

/// Writes whole blocks until the kernel refuses one and returns the number of
/// bytes accepted.
fn fill(writer: &OwnedFd) -> nix::Result<usize> {
    let data = [0u8; BLOCK];
    let mut total = 0;

    loop {
        match write(writer, &data) {
            Ok(n) => total += n,
            Err(Errno::EAGAIN) => return Ok(total),
            Err(error) => return Err(error),
        }
    }
}

/// Reads and discards up to `n` bytes.
fn drain(reader: &OwnedFd, n: usize) -> nix::Result<usize> {
    read(reader, &mut vec![0u8; n])
}

capacity calls fill on a new pipe, again after 1 byte has been read from the full pipe, and again after 4096 bytes have been read in total.

Listing 1.7 Capacity, and the space a read frees, in Go. code/go/pipefacts/main.go

// capacity fills a pipe when it is new, after 1 byte has been read, and
// after 4096 bytes have been read. Each fill reports the free space.
func capacity() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(reader)
	defer unix.Close(writer)

	steps := []struct {
		read   int
		report string
	}{
		{0, "new pipe: %d bytes accepted\n"},
		{1, "after reading 1 byte: %d bytes accepted\n"},
		{block - 1, "after reading 4096 bytes in total: %d bytes accepted\n"},
	}

	for _, step := range steps {
		if err := drain(reader, step.read); err != nil {
			return err
		}

		accepted, err := fill(writer)

		if err != nil {
			return err
		}

		fmt.Printf(step.report, accepted)
	}

	return nil
}

Listing 1.8 Capacity, and the space a read frees, in Rust. code/rust/pipefacts/src/main.rs

/// Fills a pipe when it is new, after 1 byte has been read, and after 4096
/// bytes have been read. Each fill reports the free space.
fn capacity() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;

    let steps = [
        (0, "new pipe"),
        (1, "after reading 1 byte"),
        (BLOCK - 1, "after reading 4096 bytes in total"),
    ];

    for (count, label) in steps {
        drain(&reader, count)?;

        let accepted = fill(&writer)?;
        println!("{label}: {accepted} bytes accepted");
    }

    Ok(())
}

resize changes the number of slots. fcntl with the command F_SETPIPE_SZ asks for a capacity in bytes and returns the capacity granted. The program asks for 16384 bytes, for 5000 bytes, and for 1048576 bytes, and measures the pipe after each request.

Listing 1.9 Changing the capacity of a pipe, in Go. code/go/pipefacts/main.go

// resize changes the capacity of a pipe with fcntl and measures it after
// each change.
func resize() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(reader)
	defer unix.Close(writer)

	for _, request := range []int{4 * block, 5000, 1 << 20} {
		granted, err := unix.FcntlInt(uintptr(writer), unix.F_SETPIPE_SZ, request)

		if err != nil {
			return fmt.Errorf("F_SETPIPE_SZ: %w", err)
		}

		accepted, err := fill(writer)

		if err != nil {
			return err
		}

		fmt.Printf("capacity of %d bytes requested: %d granted, %d bytes accepted\n", request, granted, accepted)

		if err := drain(reader, accepted); err != nil {
			return err
		}
	}

	return nil
}

Listing 1.10 Changing the capacity of a pipe, in Rust. code/rust/pipefacts/src/main.rs

/// Changes the capacity of a pipe with fcntl and measures it after each
/// change.
fn resize() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;

    for request in [4 * BLOCK, 5000, 1 << 20] {
        let granted = fcntl(&writer, FcntlArg::F_SETPIPE_SZ(request as i32))?;
        let accepted = fill(&writer)?;

        println!(
            "capacity of {request} bytes requested: {granted} granted, {accepted} bytes accepted"
        );

        drain(&reader, accepted)?;
    }

    Ok(())
}

endOfStream reads an empty pipe with the write end open, and again with it closed.

Listing 1.11 Reading an empty pipe, in Go. code/go/pipefacts/main.go

// endOfStream reads an empty pipe while a write end is open, and again after
// the write end has been closed.
func endOfStream() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(reader)

	buf := make([]byte, block)

	_, err = unix.Read(reader, buf)
	fmt.Printf("empty pipe, write end open: read fails with %s\n", name(err))

	unix.Close(writer)

	n, err := unix.Read(reader, buf)

	if err != nil {
		return fmt.Errorf("read: %w", err)
	}

	fmt.Printf("empty pipe, write end closed: read returns %d\n", n)

	return nil
}

Listing 1.12 Reading an empty pipe, in Rust. code/rust/pipefacts/src/main.rs

/// Reads an empty pipe while a write end is open, and again after the write
/// end has been closed.
fn end_of_stream() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;
    let mut buf = [0u8; BLOCK];

    if let Err(error) = read(&reader, &mut buf) {
        println!("empty pipe, write end open: read fails with {error:?}");
    }

    drop(writer);

    let n = read(&reader, &mut buf)?;
    println!("empty pipe, write end closed: read returns {n}");

    Ok(())
}

brokenPipe closes the read end and then writes.

Listing 1.13 Writing with no read end open, in Go. code/go/pipefacts/main.go

// brokenPipe writes to a pipe whose read end has been closed.
func brokenPipe() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(writer)

	unix.Close(reader)

	_, err = unix.Write(writer, []byte("hello"))
	fmt.Printf("read end closed: write fails with %s\n", name(err))

	return nil
}

// name returns the symbolic name of a system call error, such as EAGAIN.
func name(err error) string {
	var errno unix.Errno

	if errors.As(err, &errno) {
		return unix.ErrnoName(errno)
	}

	return fmt.Sprint(err)
}

Listing 1.14 Writing with no read end open, in Rust. code/rust/pipefacts/src/main.rs

/// Writes to a pipe whose read end has been closed.
fn broken_pipe() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;

    drop(reader);

    if let Err(error) = write(&writer, b"hello") {
        println!("read end closed: write fails with {error:?}");
    }

    Ok(())
}

The Go and Rust programs print the following on a Jetson Orin NX running Linux 5.15:

new pipe: 65536 bytes accepted
after reading 1 byte: 0 bytes accepted
after reading 4096 bytes in total: 4096 bytes accepted
capacity of 16384 bytes requested: 16384 granted, 16384 bytes accepted
capacity of 5000 bytes requested: 8192 granted, 8192 bytes accepted
capacity of 1048576 bytes requested: 1048576 granted, 1048576 bytes accepted
empty pipe, write end open: read fails with EAGAIN
empty pipe, write end closed: read returns 0
read end closed: write fails with EPIPE

In the last case the kernel also sends the writing process SIGPIPE. A signal is an asynchronous notification from the kernel to a process, and the default action for SIGPIPE is to terminate the process. The programs continue and print EPIPE because their runtimes change that default: the Go runtime handles SIGPIPE for descriptors other than 1 and 2, and a Rust program starts with SIGPIPE ignored. The pipeline program shows the default action.

Rows 2 and 3 confirm that the buffer is made of whole pages and rule out a flat byte array, since a flat array with 1 byte removed would accept 1 byte.

The buffer: a ring of pages

The kernel allocates physical memory in pages, and the pipe buffer is built from that unit. A pipe has an array of slots, 16 by default, and each slot refers to a page. Each slot also records which part of its page holds unread data: an offset to the first unread byte and a length.

Since the buffer is built from page references, an empty pipe needs no data pages and an idle pipe costs little memory. A page that already contains the data can also be placed in a slot directly: the splice system call moves page references between a file and a pipe and copies no bytes.

The slots are used as a ring, indexed by the counters head and tail:

• head is the number of pages added to the ring since the pipe was created.
• tail is the number of pages removed.

head and tail only increase. The ring maintains these invariants:

1. head - tail is the number of pages in use, and it is between 0 and the number of slots.
2. Counter value c refers to slot c mod slots. The oldest page is at tail and the newest at head - 1.
3. New bytes are appended to the newest page if it has unused space after its data. The remainder goes into new pages at head.
4. Bytes are removed from the oldest page. When its length reaches 0, the page is released and tail advances by 1.

The kernel computes the slot with a mask in place of a division: head & (ring_size - 1). The 2 are equal when ring_size is a power of 2, which is the reason a request for 5000 bytes was rounded up to 2 slots. The counters are unsigned 32-bit numbers and wrap around to 0 after about 4 billion pages. head - tail remains correct across the wrap, because unsigned subtraction wraps in the same way.

Invariants 3 and 4 explain the measurement. Removing bytes from the oldest page frees space at the front of that page, and invariant 3 places no new bytes there. The space becomes usable only when the whole page is released.

The table below traces a ring of 4 slots with 8-byte pages. The last column lists the length of each slot, with a dot for a free slot.

Implementing the ring

page and ring follow the description directly. slot applies invariant 2. Both versions allocate every page when the ring is created, which keeps the code short; the kernel allocates a page when a slot first needs one.

Listing 1.15 The imports of handpipe, in Go. code/go/handpipe/main.go

import (
	"errors"
	"fmt"
	"strings"
	"sync"
)

Listing 1.16 The imports of handpipe, in Rust. code/rust/handpipe/src/main.rs

use std::fmt;
use std::thread;

use parking_lot::{Condvar, Mutex};

Listing 1.17 The page, the ring, and its counters, in Go. code/go/handpipe/main.go

// A page is one slot of the ring: a block of memory, and the position and
// length of the bytes in it that have not been read yet.
type page struct {
	data   []byte
	offset int
	length int
}

// A ring is the storage of a pipe. head counts the pages ever put into use
// and tail counts the pages ever emptied, so head-tail pages are in use, and
// a counter's remainder after division by the number of slots is its slot.
type ring struct {
	pages []page
	head  uint
	tail  uint
}

func newRing(slots, pageSize int) *ring {
	pages := make([]page, slots)

	for i := range pages {
		pages[i].data = make([]byte, pageSize)
	}

	return &ring{pages: pages}
}

func (r *ring) empty() bool {
	return r.head == r.tail
}

func (r *ring) full() bool {
	return r.head-r.tail == uint(len(r.pages))
}

// slot returns the page that a counter refers to.
func (r *ring) slot(counter uint) *page {
	return &r.pages[counter%uint(len(r.pages))]
}

Listing 1.18 The page, the ring, and its counters, in Rust. code/rust/handpipe/src/main.rs

/// One slot of the ring: a block of memory, and the position and length of
/// the bytes in it that have not been read yet.
struct Page {
    data: Vec<u8>,
    offset: usize,
    length: usize,
}

/// The storage of a pipe. `head` counts the pages ever put into use and
/// `tail` counts the pages ever emptied, so `head - tail` pages are in use,
/// and a counter's remainder after division by the number of slots is its
/// slot.
struct Ring {
    pages: Vec<Page>,
    head: usize,
    tail: usize,
}

impl Ring {
    fn new(slots: usize, page_size: usize) -> Ring {
        let pages = (0..slots)
            .map(|_| Page {
                data: vec![0; page_size],
                offset: 0,
                length: 0,
            })
            .collect();

        Ring {
            pages,
            head: 0,
            tail: 0,
        }
    }

    fn empty(&self) -> bool {
        self.head == self.tail
    }

    fn full(&self) -> bool {
        self.head - self.tail == self.pages.len()
    }

    /// The page that a counter refers to.
    fn slot(&mut self, counter: usize) -> &mut Page {
        let slots = self.pages.len();

        &mut self.pages[counter % slots]
    }
}

put implements invariant 3: first the unused space of the newest page, then free slots. It returns the number of bytes copied, which is smaller than the request when the ring fills.

Listing 1.19 Appending bytes to the ring, in Go. code/go/handpipe/main.go

// put copies as much of data as the ring has room for and returns the number
// of bytes it copied. The bytes go first into the unused end of the newest
// page, then into free slots, one page each.
func (r *ring) put(data []byte) int {
	copied := 0

	if !r.empty() {
		newest := r.slot(r.head - 1)
		end := newest.offset + newest.length

		n := copy(newest.data[end:], data)
		newest.length += n
		copied += n
	}

	for copied < len(data) && !r.full() {
		next := r.slot(r.head)
		next.offset = 0
		next.length = copy(next.data, data[copied:])

		copied += next.length
		r.head++
	}

	return copied
}

Listing 1.20 Appending bytes to the ring, in Rust. code/rust/handpipe/src/main.rs

/// Copies as many bytes as both slices allow and returns the count.
fn copy(to: &mut [u8], from: &[u8]) -> usize {
    let n = to.len().min(from.len());
    to[..n].copy_from_slice(&from[..n]);

    n
}

impl Ring {
    /// Copies as much of `data` as the ring has room for and returns the
    /// number of bytes it copied. The bytes go first into the unused end of
    /// the newest page, then into free slots, one page each.
    fn put(&mut self, data: &[u8]) -> usize {
        let mut copied = 0;

        if !self.empty() {
            let newest = self.slot(self.head - 1);
            let end = newest.offset + newest.length;

            let n = copy(&mut newest.data[end..], data);
            newest.length += n;
            copied += n;
        }

        while copied < data.len() && !self.full() {
            let next = self.slot(self.head);
            next.offset = 0;
            next.length = copy(&mut next.data, &data[copied..]);

            copied += next.length;
            self.head += 1;
        }

        copied
    }
}

take implements invariant 4. It advances offset in the oldest page and increments tail when the page is exhausted.

Listing 1.21 Removing the oldest bytes from the ring, in Go. code/go/handpipe/main.go

// take moves up to len(buf) of the oldest bytes into buf and returns the
// number it moved. A slot becomes free when the last byte of its page has
// been taken.
func (r *ring) take(buf []byte) int {
	moved := 0

	for moved < len(buf) && !r.empty() {
		oldest := r.slot(r.tail)

		n := copy(buf[moved:], oldest.data[oldest.offset:oldest.offset+oldest.length])
		oldest.offset += n
		oldest.length -= n
		moved += n

		if oldest.length == 0 {
			r.tail++
		}
	}

	return moved
}

Listing 1.22 Removing the oldest bytes from the ring, in Rust. code/rust/handpipe/src/main.rs

impl Ring {
    /// Moves up to `buf.len()` of the oldest bytes into `buf` and returns the
    /// number it moved. A slot becomes free when the last byte of its page
    /// has been taken.
    fn take(&mut self, buf: &mut [u8]) -> usize {
        let mut moved = 0;

        while moved < buf.len() && !self.empty() {
            let oldest = self.slot(self.tail);
            let unread = oldest.offset..oldest.offset + oldest.length;

            let n = copy(&mut buf[moved..], &oldest.data[unread]);
            oldest.offset += n;
            oldest.length -= n;
            moved += n;

            if oldest.length == 0 {
                self.tail += 1;
            }
        }

        moved
    }
}

smallRing applies the calls of the trace and prints the ring after each.

Listing 1.23 Running the trace, in Go. code/go/handpipe/main.go

// picture renders the ring as one cell per slot: the number of unread bytes
// in its page, or a dot when the slot is free.
func (r *ring) picture() string {
	cells := make([]string, len(r.pages))

	for i := range cells {
		cells[i] = "."
	}

	for counter := r.tail; counter != r.head; counter++ {
		cells[counter%uint(len(r.pages))] = fmt.Sprint(r.slot(counter).length)
	}

	return "[" + strings.Join(cells, " ") + "]"
}

// smallRing applies the calls of the trace to a ring of 4 pages of 8 bytes
// and prints the ring after each.
func smallRing() {
	r := newRing(4, 8)
	buf := make([]byte, 32)

	calls := []struct {
		op string
		n  int
	}{{"put", 12}, {"take", 3}, {"put", 30}, {"take", 5}, {"put", 30}}

	fmt.Println("ring of 4 pages of 8 bytes")

	for _, call := range calls {
		var done int

		switch call.op {
		case "put":
			done = r.put(buf[:call.n])
		case "take":
			done = r.take(buf[:call.n])
		}

		fmt.Printf("%-4s %2d: %2d bytes   head %d  tail %d   %s\n", call.op, call.n, done, r.head, r.tail, r.picture())
	}
}

Listing 1.24 Running the trace, in Rust. code/rust/handpipe/src/main.rs

impl Ring {
    /// Renders the ring as one cell per slot: the unread bytes in its page,
    /// or a dot for a free slot.
    fn picture(&self) -> String {
        let slots = self.pages.len();
        let mut cells = vec![".".to_string(); slots];

        for counter in self.tail..self.head {
            cells[counter % slots] = self.pages[counter % slots].length.to_string();
        }

        format!("[{}]", cells.join(" "))
    }
}

/// Applies the calls of the trace to a ring of 4 pages of 8 bytes and prints
/// the ring after each.
fn small_ring() {
    let mut ring = Ring::new(4, 8);
    let mut buf = [0u8; 32];

    let calls = [
        ("put", 12),
        ("take", 3),
        ("put", 30),
        ("take", 5),
        ("put", 30),
    ];

    println!("ring of 4 pages of 8 bytes");

    for (op, n) in calls {
        let done = match op {
            "put" => ring.put(&buf[..n]),
            _ => ring.take(&mut buf[..n]),
        };

        println!(
            "{op:<4} {n:2}: {done:2} bytes   head {}  tail {}   {}",
            ring.head,
            ring.tail,
            ring.picture()
        );
    }
}

ring of 4 pages of 8 bytes
put  12: 12 bytes   head 2  tail 0   [8 4 . .]
take  3:  3 bytes   head 2  tail 0   [5 4 . .]
put  30: 20 bytes   head 4  tail 0   [5 8 8 8]
take  5:  5 bytes   head 4  tail 1   [. 8 8 8]
put  30:  8 bytes   head 5  tail 1   [8 8 8 8]

The output agrees with the table.

Checking the model against the kernel

kernelSize repeats the capacity measurement of pipefacts on a ring with the kernel’s dimensions, 16 pages of 4096 bytes.

Listing 1.25 The capacity measurement on a ring of 16 pages, in Go. code/go/handpipe/main.go

// fill puts whole blocks into the ring until it accepts none and returns the
// number of bytes accepted.
func fill(r *ring, block []byte) int {
	total := 0

	for {
		n := r.put(block)

		if n == 0 {
			return total
		}

		total += n
	}
}

// kernelSize repeats the capacity measurement of pipefacts on a ring with
// the kernel's dimensions: 16 pages of 4096 bytes.
func kernelSize() {
	const pageSize = 4096

	r := newRing(16, pageSize)
	block := make([]byte, pageSize)

	steps := []struct {
		read   int
		report string
	}{
		{0, "new pipe: %d bytes accepted\n"},
		{1, "after reading 1 byte: %d bytes accepted\n"},
		{pageSize - 1, "after reading 4096 bytes in total: %d bytes accepted\n"},
	}

	for _, step := range steps {
		r.take(block[:step.read])
		fmt.Printf(step.report, fill(r, block))
	}
}

Listing 1.26 The capacity measurement on a ring of 16 pages, in Rust. code/rust/handpipe/src/main.rs

/// Puts whole blocks into the ring until it accepts none and returns the
/// number of bytes accepted.
fn fill(ring: &mut Ring, block: &[u8]) -> usize {
    let mut total = 0;

    loop {
        let n = ring.put(block);

        if n == 0 {
            return total;
        }

        total += n;
    }
}

/// Repeats the capacity measurement of pipefacts on a ring with the kernel's
/// dimensions: 16 pages of 4096 bytes.
fn kernel_size() {
    const PAGE_SIZE: usize = 4096;

    let mut ring = Ring::new(16, PAGE_SIZE);
    let mut block = [0u8; PAGE_SIZE];

    let steps = [
        (0, "new pipe"),
        (1, "after reading 1 byte"),
        (PAGE_SIZE - 1, "after reading 4096 bytes in total"),
    ];

    for (count, label) in steps {
        ring.take(&mut block[..count]);

        let accepted = fill(&mut ring, &block);
        println!("{label}: {accepted} bytes accepted");
    }
}

new pipe: 65536 bytes accepted
after reading 1 byte: 0 bytes accepted
after reading 4096 bytes in total: 4096 bytes accepted

These lines are identical to lines 1 to 3 of the pipefacts output. The ring of pages reproduces the kernel’s capacity and its page-sized release of space.

Blocking

The ring is a single-threaded data structure. A pipe adds safe access from a reader and a writer running concurrently, and a defined behaviour when a call cannot make progress. Figure 1.7 lists the cases of each call.

The blocking cases provide flow control. If seq produces output faster than grep consumes it, the ring fills and seq is suspended inside write. When grep releases a page, seq resumes. The writer’s rate is therefore limited to the reader’s rate, and the memory between them stays at 16 pages for a stream of any length. Neither program contains code for this; it follows from the semantics of write.

In addition to the ring, the implementation needs:

• A lock, which gives a thread exclusive access to the ring while it holds the lock.
• The condition variables readable and writable. A condition variable is a queue on which a thread sleeps until another thread wakes it. Waiting releases the lock and suspends the thread atomically, and reacquires the lock before returning. Readers wait on readable and writers on writable. The kernel’s equivalents are the wait queues rd_wait and wr_wait.
• The counts readers and writers of the open ends.

Listing 1.27 The pipe: a ring, a lock, wait queues, and the open ends, in Go. code/go/handpipe/main.go

// errBrokenPipe is what a write returns when no reader is left.
var errBrokenPipe = errors.New("EPIPE")

// A pipe is a ring shared by a reader and a writer goroutine. readable is where a reader
// sleeps while the ring is empty, and writable is where a writer sleeps while
// the ring is full. readers and writers count the open ends.
type pipe struct {
	mu       sync.Mutex
	readable *sync.Cond
	writable *sync.Cond
	ring     *ring
	readers  int
	writers  int
}

func newPipe(slots, pageSize int) *pipe {
	p := &pipe{ring: newRing(slots, pageSize), readers: 1, writers: 1}
	p.readable = sync.NewCond(&p.mu)
	p.writable = sync.NewCond(&p.mu)

	return p
}

Listing 1.28 The pipe: a ring, a lock, wait queues, and the open ends, in Rust. code/rust/handpipe/src/main.rs

/// What a write returns when no reader is left.
#[derive(Debug)]
struct BrokenPipe;

impl fmt::Display for BrokenPipe {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("EPIPE")
    }
}

/// What the lock guards: the ring and the count of the open ends.
struct State {
    ring: Ring,
    readers: usize,
    writers: usize,
}

/// A ring shared by a reader and a writer thread. `readable` is where a reader sleeps while
/// the ring is empty, and `writable` is where a writer sleeps while the ring
/// is full.
struct Pipe {
    state: Mutex<State>,
    readable: Condvar,
    writable: Condvar,
}

impl Pipe {
    fn new(slots: usize, page_size: usize) -> Pipe {
        let state = State {
            ring: Ring::new(slots, page_size),
            readers: 1,
            writers: 1,
        };

        Pipe {
            state: Mutex::new(state),
            readable: Condvar::new(),
            writable: Condvar::new(),
        }
    }
}

write loops until all of data is in the ring. On each pass it checks for a reader, appends what fits, and either wakes the readers or waits on writable.

A woken thread must test its condition again before proceeding. A wake-up indicates that the state changed, and the change may be unrelated to the condition the thread waits for: in row 2 of the trace, a take of 3 bytes freed no slot. For this reason the wait is inside the loop, and the writer calls put again after every wake-up.

Listing 1.29 Blocking write, in Go. code/go/handpipe/main.go

// write returns when every byte of data is in the ring, or when no reader is
// left. While the ring is full it sleeps on writable.
func (p *pipe) write(data []byte) (int, error) {
	p.mu.Lock()
	defer p.mu.Unlock()

	written := 0

	for written < len(data) {
		if p.readers == 0 {
			return written, errBrokenPipe
		}

		n := p.ring.put(data[written:])
		written += n

		if n == 0 {
			p.writable.Wait()
			continue
		}

		p.readable.Broadcast()
	}

	return written, nil
}

Listing 1.30 Blocking write, in Rust. code/rust/handpipe/src/main.rs

impl Pipe {
    /// Returns when every byte of `data` is in the ring, or when no reader is
    /// left. While the ring is full it sleeps on `writable`.
    fn write(&self, data: &[u8]) -> Result<usize, BrokenPipe> {
        let mut state = self.state.lock();
        let mut written = 0;

        while written < data.len() {
            if state.readers == 0 {
                return Err(BrokenPipe);
            }

            let n = state.ring.put(&data[written..]);
            written += n;

            if n == 0 {
                self.writable.wait(&mut state);
                continue;
            }

            self.readable.notify_all();
        }

        Ok(written)
    }
}

read waits while the ring is empty and a writer exists. If it wakes with the ring still empty, writers is 0, take returns 0, and so does read. A read returns as soon as at least 1 byte is available, so the caller may receive fewer bytes than it asked for and must be prepared to call read again.

Listing 1.31 Blocking read, in Go. code/go/handpipe/main.go

// read returns as soon as it has moved at least one byte into buf. While the
// ring is empty and a writer exists it sleeps on readable. It returns 0 when
// the ring is empty and no writer is left: the end of the stream.
func (p *pipe) read(buf []byte) int {
	p.mu.Lock()
	defer p.mu.Unlock()

	for p.ring.empty() && p.writers > 0 {
		p.readable.Wait()
	}

	n := p.ring.take(buf)

	if n > 0 {
		p.writable.Broadcast()
	}

	return n
}

Listing 1.32 Blocking read, in Rust. code/rust/handpipe/src/main.rs

impl Pipe {
    /// Returns as soon as it has moved at least one byte into `buf`. While
    /// the ring is empty and a writer exists it sleeps on `readable`. It
    /// returns 0 when the ring is empty and no writer is left: the end of the
    /// stream.
    fn read(&self, buf: &mut [u8]) -> usize {
        let mut state = self.state.lock();

        self.readable
            .wait_while(&mut state, |state| state.ring.empty() && state.writers > 0);

        let n = state.ring.take(buf);

        if n > 0 {
            self.writable.notify_all();
        }

        n
    }
}

Closing an end decrements its count and wakes the opposite side. Without the wake-up, a reader already waiting on an empty ring would remain suspended after writers reached 0.

Listing 1.33 Closing an end, in Go. code/go/handpipe/main.go

// closeWriter gives up a write end. A reader asleep on an empty ring wakes
// and finds that no writer is left.
func (p *pipe) closeWriter() {
	p.mu.Lock()
	defer p.mu.Unlock()

	p.writers--
	p.readable.Broadcast()
}

// closeReader gives up a read end. A writer asleep on a full ring wakes and
// finds that no reader is left.
func (p *pipe) closeReader() {
	p.mu.Lock()
	defer p.mu.Unlock()

	p.readers--
	p.writable.Broadcast()
}

Listing 1.34 Closing an end, in Rust. code/rust/handpipe/src/main.rs

impl Pipe {
    /// Gives up a write end. A reader asleep on an empty ring wakes and finds
    /// that no writer is left.
    fn close_writer(&self) {
        self.state.lock().writers -= 1;
        self.readable.notify_all();
    }

    /// Gives up a read end. A writer asleep on a full ring wakes and finds
    /// that no reader is left.
    fn close_reader(&self) {
        self.state.lock().readers -= 1;
        self.writable.notify_all();
    }
}

Animation 1.1 runs a writer and a reader on the 4-slot ring. The writer’s second call writes 30 bytes. It blocks after 20 bytes and again after 28, and returns when all 30 bytes are in the ring.

transfer tests the complete pipe. A writer thread sends 1048576 bytes in writes of 100000 bytes. Each write is larger than the ring’s capacity, so each one blocks several times. The reader verifies the order of the bytes, then reads again after the write end is closed. The read end is then closed and another write is attempted.

Listing 1.35 One megabyte through a 16-page ring, in Go. code/go/handpipe/main.go

// transfer sends one megabyte through a pipe of the kernel's size. Byte i of
// the stream has the value i mod 256, which lets the reader verify the order.
func transfer() {
	const total = 1 << 20

	p := newPipe(16, 4096)

	go func() {
		defer p.closeWriter()

		chunk := make([]byte, 100_000)

		for sent := 0; sent < total; sent += len(chunk) {
			chunk = chunk[:min(len(chunk), total-sent)]

			for i := range chunk {
				chunk[i] = byte(sent + i)
			}

			if _, err := p.write(chunk); err != nil {
				return
			}
		}
	}()

	buf := make([]byte, 30_000)
	received, inOrder := 0, true

	for n := p.read(buf); n > 0; n = p.read(buf) {
		for i, b := range buf[:n] {
			inOrder = inOrder && b == byte(received+i)
		}

		received += n
	}

	fmt.Printf("1048576 bytes written: %d bytes read, in order: %v\n", received, inOrder)
	fmt.Printf("write end closed: read returns %d\n", p.read(buf))

	p.closeReader()

	_, err := p.write([]byte("hello"))
	fmt.Printf("read end closed: write fails with %v\n", err)
}

Listing 1.36 One megabyte through a 16-page ring, in Rust. code/rust/handpipe/src/main.rs

/// Sends one megabyte through a pipe of the kernel's size. Byte `i` of the
/// stream has the value `i` mod 256, which lets the reader verify the order.
fn transfer() {
    const TOTAL: usize = 1 << 20;

    let pipe = Pipe::new(16, 4096);
    let mut buf = vec![0u8; 30_000];

    thread::scope(|scope| {
        scope.spawn(|| {
            let mut chunk = vec![0u8; 100_000];

            for sent in (0..TOTAL).step_by(chunk.len()) {
                chunk.truncate(TOTAL - sent);

                for (i, byte) in chunk.iter_mut().enumerate() {
                    *byte = (sent + i) as u8;
                }

                if pipe.write(&chunk).is_err() {
                    break;
                }
            }

            pipe.close_writer();
        });

        let (mut received, mut in_order) = (0, true);

        loop {
            let n = pipe.read(&mut buf);

            if n == 0 {
                break;
            }

            for (i, byte) in buf[..n].iter().enumerate() {
                in_order = in_order && *byte == (received + i) as u8;
            }

            received += n;
        }

        println!("1048576 bytes written: {received} bytes read, in order: {in_order}");
    });

    println!("write end closed: read returns {}", pipe.read(&mut buf));

    pipe.close_reader();

    if let Err(error) = pipe.write(b"hello") {
        println!("read end closed: write fails with {error}");
    }
}

1048576 bytes written: 1048576 bytes read, in order: true
write end closed: read returns 0
read end closed: write fails with EPIPE

All bytes arrived in order through 65536 bytes of buffer. The read that returns 0 and the write that fails with EPIPE match the pipefacts output.

The end of a stream

readers and writers in struct pipe_inode_info count open ends, and both rules for the end of a stream are tests of these counts.

The counts change when an end is opened or closed, and “closed” has a precise meaning. An end of a pipe is a struct file, and several descriptors can refer to the same struct file: dup creates another one, and fork gives the child a copy of every descriptor of the parent. The kernel releases the struct file, and lowers readers or writers, when the last of those descriptors is closed. A single forgotten descriptor in any process keeps the end open.

Figure 1.8 follows the counts through the life of the pipeline a | b.

The figure counts descriptors, as a programmer has to. Row 3 is the step that programs get wrong. After the forks, 3 processes hold a descriptor for the write end. The reader b holds one itself, and if it does not close it, b waits for the end of a stream whose last writer is b.

Building a pipeline

pipeline runs seq 1 100000 | grep 7 | wc -l with kernel pipes. The kernel’s rules for descriptors determine how it must manage them.

1. Descriptors are inherited. When a process (the parent) starts another (the child), the child receives copies of the parent’s descriptors. A copy refers to the same pipe end as the original.
2. An end is closed when its last descriptor is closed. The kernel counts the descriptors that refer to each end. read returns 0 only when the count for the write end is 0.

Suppose the parent creates a pipe, starts seq with the write end and grep with the read end, and keeps its own descriptors open. When seq exits, the write end still has an open descriptor, the parent’s. grep then blocks in read indefinitely and the pipeline does not finish. The rule for the parent follows: after starting a child, close the parent’s copy of every pipe end that was passed to it.

O_CLOEXEC enforces the same rule for descriptors that were not passed deliberately. The kernel closes a descriptor carrying this flag when the process executes a new program. Go’s os.Pipe and Rust’s io::pipe both set it, so each command receives its own ends and no descriptors for the other pipes.

start creates a pipe for each command. Each command writes to its own pipe and reads from the previous pipe, and the read end of the newest pipe is carried to the next iteration. The parent’s copies are closed in each iteration: explicitly in Go, and in Rust by ownership, since the ends are moved into the Command, which is dropped at the end of the statement.

Listing 1.37 The imports of pipeline, in Go. code/go/pipeline/main.go

import (
	"fmt"
	"io"
	"log"
	"os"
	"os/exec"
	"strings"
	"syscall"
)

Listing 1.38 The imports of pipeline, in Rust. code/rust/pipeline/src/main.rs

use std::io::{self, PipeReader, Read};
use std::os::unix::process::ExitStatusExt;
use std::process::{Child, Command, Stdio};

Listing 1.39 Starting the commands, each with its own pipe, in Go. code/go/pipeline/main.go

// start runs the commands of a pipeline. Command i writes to pipe i and reads
// from pipe i-1; the first command reads /dev/null. It returns the commands
// and the read end of the last pipe.
func start(stages [][]string) ([]*exec.Cmd, *os.File, error) {
	var input *os.File
	commands := make([]*exec.Cmd, 0, len(stages))

	for _, stage := range stages {
		reader, writer, err := os.Pipe()

		if err != nil {
			return nil, nil, err
		}

		command := exec.Command(stage[0], stage[1:]...)
		command.Stdout = writer

		if input != nil {
			command.Stdin = input
		}

		err = command.Start()

		// The child holds its own copies of both ends. Closing the parent's
		// copies leaves the child's as the only ones.
		writer.Close()

		if input != nil {
			input.Close()
		}

		if err != nil {
			reader.Close()

			return nil, nil, fmt.Errorf("%s: %w", stage[0], err)
		}

		commands, input = append(commands, command), reader
	}

	return commands, input, nil
}

Listing 1.40 Starting the commands, each with its own pipe, in Rust. code/rust/pipeline/src/main.rs

/// Runs the commands of a pipeline. Command `i` writes to pipe `i` and reads
/// from pipe `i - 1`; the first command reads /dev/null. Returns the children
/// and the read end of the last pipe.
fn start(stages: &[Vec<&str>]) -> io::Result<(Vec<Child>, PipeReader)> {
    let mut input = None;
    let mut children = Vec::with_capacity(stages.len());

    for stage in stages {
        let (reader, writer) = io::pipe()?;
        let stdin = input.take().map_or_else(Stdio::null, Stdio::from);

        // The `Command` owns the parent's copies of both ends. It is dropped
        // when this statement ends, which closes them and leaves the child's
        // copies as the only ones.
        let child = Command::new(stage[0])
            .args(&stage[1..])
            .stdin(stdin)
            .stdout(writer)
            .spawn()?;

        children.push(child);
        input = Some(reader);
    }

    let output = input.ok_or(io::ErrorKind::InvalidInput)?;

    Ok((children, output))
}

The kernel records how each process terminated: with an exit status chosen by the process, where 0 means success by convention, or by a signal. The parent retrieves the record with the wait system call.

Listing 1.41 Reporting how a command terminated, in Go. code/go/pipeline/main.go

// ending waits for a command and reports how it terminated: with an exit
// status, or by a signal.
func ending(command *exec.Cmd) string {
	command.Wait()

	state := command.ProcessState

	if status, ok := state.Sys().(syscall.WaitStatus); ok && status.Signaled() {
		return fmt.Sprintf("killed by signal %d", status.Signal())
	}

	return fmt.Sprintf("exited with status %d", state.ExitCode())
}

Listing 1.42 Reporting how a command terminated, in Rust. code/rust/pipeline/src/main.rs

/// Waits for a command and reports how it terminated: with an exit status,
/// or by a signal.
fn ending(child: &mut Child) -> io::Result<String> {
    let status = child.wait()?;

    let text = match (status.signal(), status.code()) {
        (Some(signal), _) => format!("killed by signal {signal}"),
        (None, Some(code)) => format!("exited with status {code}"),
        (None, None) => status.to_string(),
    };

    Ok(text)
}

run reads the last pipe until read returns 0. That occurs when the last command has exited and no descriptor for the write end remains, which depends on start having closed the parent’s copy.

Listing 1.43 Running a pipeline, in Go. code/go/pipeline/main.go

// run starts a pipeline, reads the last command's output to the end of the
// stream, and prints the output and each command's termination.
func run(line string) error {
	var stages [][]string

	for _, stage := range strings.Split(line, " | ") {
		stages = append(stages, strings.Fields(stage))
	}

	commands, output, err := start(stages)

	if err != nil {
		return err
	}

	defer output.Close()

	text, err := io.ReadAll(output)

	if err != nil {
		return err
	}

	fmt.Println(line)
	fmt.Printf("  output: %s\n", strings.Join(strings.Fields(string(text)), " "))

	for i, command := range commands {
		fmt.Printf("  %s: %s\n", stages[i][0], ending(command))
	}

	return nil
}

Listing 1.44 Running a pipeline, in Rust. code/rust/pipeline/src/main.rs

/// Starts a pipeline, reads the last command's output to the end of the
/// stream, and prints the output and each command's termination.
fn run(line: &str) -> io::Result<()> {
    let stages: Vec<Vec<&str>> = line
        .split(" | ")
        .map(|stage| stage.split_whitespace().collect())
        .collect();

    let (mut children, mut output) = start(&stages)?;

    let mut text = String::new();
    output.read_to_string(&mut text)?;

    let words: Vec<&str> = text.split_whitespace().collect();

    println!("{line}");
    println!("  output: {}", words.join(" "));

    for (stage, child) in stages.iter().zip(&mut children) {
        println!("  {}: {}", stage[0], ending(child)?);
    }

    Ok(())
}

seq 1 100000 | grep 7 | wc -l
  output: 40951
  seq: exited with status 0
  grep: exited with status 0
  wc: exited with status 0
yes | head -n 3
  output: y y y
  yes: killed by signal 13
  head: exited with status 0

In the first pipeline, each command reached the end of its input when the previous command exited, and all of them exited with status 0. The result can be verified independently. Of the 100000 5-digit strings from 00000 to 99999, 9⁵ = 59049 contain no 7. The range 1 to 100000 replaces 00000 with 100000, and neither contains a 7, so 100000 - 59049 = 40951 numbers in the range contain a 7.

The second pipeline terminates a writer that has no end of its own. yes writes the line y in a loop. head -n 3 exits after 3 lines, which closes the only read end of the pipe. The next write by yes raises SIGPIPE, signal 13, and the default action terminates the process. In a longer pipeline this propagates from right to left: each command that exits closes a read end, and the command before it is terminated on its next write.

Named pipes

The pipes so far have no name, so only processes that inherited the descriptors can use them. A named pipe, or FIFO, is the same kernel object with an entry in the file system, created by mkfifo. Unrelated processes open it by path, one for writing and one for reading, and the bytes pass through a struct pipe_inode_info exactly as before. Nothing is written to the disk. The p at the start of the permissions marks the type.

$ mkfifo /tmp/x/queue
$ ls -l /tmp/x/queue
prw-rw-r-- /tmp/x/queue
$ echo hello > /tmp/x/queue &
$ cat /tmp/x/queue
hello

An open of a FIFO for writing blocks until some process opens it for reading, and the reverse, which is why the echo above is started in the background.

Inspecting pipes on a live system

A pipeline that appears to hang is usually diagnosed with the first command: find the process that is blocked, read its descriptors, and look for another process that still holds the write end of the same pipe number.

The complete programs

Listing 1.45 The complete pipefacts, in Go. code/go/pipefacts/main.go

// Command pipefacts measures a kernel pipe: its capacity,
// the unit in which a read frees space, how a reader observes the end of the
// stream, and the result of a write when no reader exists.
//
// Run it from code/go with:
//
//	go run ./pipefacts
package main

import (
	"errors"
	"fmt"
	"log"

	"golang.org/x/sys/unix"
)


// block is the size of every write the program attempts: one page.
const block = 4096

// newPipe creates a pipe in non-blocking mode: a read or a write that cannot
// proceed returns EAGAIN where it would otherwise suspend the caller.
func newPipe() (reader, writer int, err error) {
	var fds [2]int

	if err := unix.Pipe2(fds[:], unix.O_NONBLOCK|unix.O_CLOEXEC); err != nil {
		return 0, 0, fmt.Errorf("pipe2: %w", err)
	}

	return fds[0], fds[1], nil
}


// fill writes whole blocks until the kernel refuses one and returns the
// number of bytes accepted.
func fill(writer int) (int, error) {
	data := make([]byte, block)
	total := 0

	for {
		n, err := unix.Write(writer, data)

		if errors.Is(err, unix.EAGAIN) {
			return total, nil
		}

		if err != nil {
			return total, fmt.Errorf("write: %w", err)
		}

		total += n
	}
}

// drain reads and discards up to n bytes.
func drain(reader, n int) error {
	_, err := unix.Read(reader, make([]byte, n))

	return err
}


// capacity fills a pipe when it is new, after 1 byte has been read, and
// after 4096 bytes have been read. Each fill reports the free space.
func capacity() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(reader)
	defer unix.Close(writer)

	steps := []struct {
		read   int
		report string
	}{
		{0, "new pipe: %d bytes accepted\n"},
		{1, "after reading 1 byte: %d bytes accepted\n"},
		{block - 1, "after reading 4096 bytes in total: %d bytes accepted\n"},
	}

	for _, step := range steps {
		if err := drain(reader, step.read); err != nil {
			return err
		}

		accepted, err := fill(writer)

		if err != nil {
			return err
		}

		fmt.Printf(step.report, accepted)
	}

	return nil
}


// resize changes the capacity of a pipe with fcntl and measures it after
// each change.
func resize() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(reader)
	defer unix.Close(writer)

	for _, request := range []int{4 * block, 5000, 1 << 20} {
		granted, err := unix.FcntlInt(uintptr(writer), unix.F_SETPIPE_SZ, request)

		if err != nil {
			return fmt.Errorf("F_SETPIPE_SZ: %w", err)
		}

		accepted, err := fill(writer)

		if err != nil {
			return err
		}

		fmt.Printf("capacity of %d bytes requested: %d granted, %d bytes accepted\n", request, granted, accepted)

		if err := drain(reader, accepted); err != nil {
			return err
		}
	}

	return nil
}


// endOfStream reads an empty pipe while a write end is open, and again after
// the write end has been closed.
func endOfStream() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(reader)

	buf := make([]byte, block)

	_, err = unix.Read(reader, buf)
	fmt.Printf("empty pipe, write end open: read fails with %s\n", name(err))

	unix.Close(writer)

	n, err := unix.Read(reader, buf)

	if err != nil {
		return fmt.Errorf("read: %w", err)
	}

	fmt.Printf("empty pipe, write end closed: read returns %d\n", n)

	return nil
}


// brokenPipe writes to a pipe whose read end has been closed.
func brokenPipe() error {
	reader, writer, err := newPipe()

	if err != nil {
		return err
	}

	defer unix.Close(writer)

	unix.Close(reader)

	_, err = unix.Write(writer, []byte("hello"))
	fmt.Printf("read end closed: write fails with %s\n", name(err))

	return nil
}

// name returns the symbolic name of a system call error, such as EAGAIN.
func name(err error) string {
	var errno unix.Errno

	if errors.As(err, &errno) {
		return unix.ErrnoName(errno)
	}

	return fmt.Sprint(err)
}


func main() {
	for _, experiment := range []func() error{capacity, resize, endOfStream, brokenPipe} {
		if err := experiment(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 1.46 The complete pipefacts, in Rust. code/rust/pipefacts/src/main.rs

//! Measures a kernel pipe: its capacity, the unit in which
//! a read frees space, how a reader observes the end of the stream, and the
//! result of a write when no reader exists.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p pipefacts
//! ```

use std::os::fd::OwnedFd;

use nix::errno::Errno;
use nix::fcntl::{FcntlArg, OFlag, fcntl};
use nix::unistd::{pipe2, read, write};

/// The size of every write the program attempts: one page.
const BLOCK: usize = 4096;

/// Creates a pipe in non-blocking mode: a read or a write that cannot proceed
/// returns `EAGAIN` where it would otherwise suspend the caller. The read end
/// is first.
fn new_pipe() -> nix::Result<(OwnedFd, OwnedFd)> {
    pipe2(OFlag::O_NONBLOCK | OFlag::O_CLOEXEC)
}

/// Writes whole blocks until the kernel refuses one and returns the number of
/// bytes accepted.
fn fill(writer: &OwnedFd) -> nix::Result<usize> {
    let data = [0u8; BLOCK];
    let mut total = 0;

    loop {
        match write(writer, &data) {
            Ok(n) => total += n,
            Err(Errno::EAGAIN) => return Ok(total),
            Err(error) => return Err(error),
        }
    }
}

/// Reads and discards up to `n` bytes.
fn drain(reader: &OwnedFd, n: usize) -> nix::Result<usize> {
    read(reader, &mut vec![0u8; n])
}

/// Fills a pipe when it is new, after 1 byte has been read, and after 4096
/// bytes have been read. Each fill reports the free space.
fn capacity() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;

    let steps = [
        (0, "new pipe"),
        (1, "after reading 1 byte"),
        (BLOCK - 1, "after reading 4096 bytes in total"),
    ];

    for (count, label) in steps {
        drain(&reader, count)?;

        let accepted = fill(&writer)?;
        println!("{label}: {accepted} bytes accepted");
    }

    Ok(())
}

/// Changes the capacity of a pipe with fcntl and measures it after each
/// change.
fn resize() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;

    for request in [4 * BLOCK, 5000, 1 << 20] {
        let granted = fcntl(&writer, FcntlArg::F_SETPIPE_SZ(request as i32))?;
        let accepted = fill(&writer)?;

        println!(
            "capacity of {request} bytes requested: {granted} granted, {accepted} bytes accepted"
        );

        drain(&reader, accepted)?;
    }

    Ok(())
}

/// Reads an empty pipe while a write end is open, and again after the write
/// end has been closed.
fn end_of_stream() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;
    let mut buf = [0u8; BLOCK];

    if let Err(error) = read(&reader, &mut buf) {
        println!("empty pipe, write end open: read fails with {error:?}");
    }

    drop(writer);

    let n = read(&reader, &mut buf)?;
    println!("empty pipe, write end closed: read returns {n}");

    Ok(())
}

/// Writes to a pipe whose read end has been closed.
fn broken_pipe() -> nix::Result<()> {
    let (reader, writer) = new_pipe()?;

    drop(reader);

    if let Err(error) = write(&writer, b"hello") {
        println!("read end closed: write fails with {error:?}");
    }

    Ok(())
}

fn main() -> nix::Result<()> {
    capacity()?;
    resize()?;
    end_of_stream()?;
    broken_pipe()
}

Listing 1.47 The complete handpipe, in Go. code/go/handpipe/main.go

// Command handpipe builds a pipe by hand, with the same parts the kernel
// uses: a ring of pages for the bytes, a lock, a place for a reader to sleep
// and a place for a writer to sleep, and a count of the open ends.
//
// It traces a small ring call by call, repeats the capacity measurement of
// pipefacts on a ring of the kernel's size, and sends 1048576 bytes from a
// writer goroutine to a reader.
//
// Run it from code/go with:
//
//	go run ./handpipe
package main

import (
	"errors"
	"fmt"
	"strings"
	"sync"
)


// A page is one slot of the ring: a block of memory, and the position and
// length of the bytes in it that have not been read yet.
type page struct {
	data   []byte
	offset int
	length int
}

// A ring is the storage of a pipe. head counts the pages ever put into use
// and tail counts the pages ever emptied, so head-tail pages are in use, and
// a counter's remainder after division by the number of slots is its slot.
type ring struct {
	pages []page
	head  uint
	tail  uint
}

func newRing(slots, pageSize int) *ring {
	pages := make([]page, slots)

	for i := range pages {
		pages[i].data = make([]byte, pageSize)
	}

	return &ring{pages: pages}
}

func (r *ring) empty() bool {
	return r.head == r.tail
}

func (r *ring) full() bool {
	return r.head-r.tail == uint(len(r.pages))
}

// slot returns the page that a counter refers to.
func (r *ring) slot(counter uint) *page {
	return &r.pages[counter%uint(len(r.pages))]
}


// put copies as much of data as the ring has room for and returns the number
// of bytes it copied. The bytes go first into the unused end of the newest
// page, then into free slots, one page each.
func (r *ring) put(data []byte) int {
	copied := 0

	if !r.empty() {
		newest := r.slot(r.head - 1)
		end := newest.offset + newest.length

		n := copy(newest.data[end:], data)
		newest.length += n
		copied += n
	}

	for copied < len(data) && !r.full() {
		next := r.slot(r.head)
		next.offset = 0
		next.length = copy(next.data, data[copied:])

		copied += next.length
		r.head++
	}

	return copied
}


// take moves up to len(buf) of the oldest bytes into buf and returns the
// number it moved. A slot becomes free when the last byte of its page has
// been taken.
func (r *ring) take(buf []byte) int {
	moved := 0

	for moved < len(buf) && !r.empty() {
		oldest := r.slot(r.tail)

		n := copy(buf[moved:], oldest.data[oldest.offset:oldest.offset+oldest.length])
		oldest.offset += n
		oldest.length -= n
		moved += n

		if oldest.length == 0 {
			r.tail++
		}
	}

	return moved
}


// errBrokenPipe is what a write returns when no reader is left.
var errBrokenPipe = errors.New("EPIPE")

// A pipe is a ring shared by a reader and a writer goroutine. readable is where a reader
// sleeps while the ring is empty, and writable is where a writer sleeps while
// the ring is full. readers and writers count the open ends.
type pipe struct {
	mu       sync.Mutex
	readable *sync.Cond
	writable *sync.Cond
	ring     *ring
	readers  int
	writers  int
}

func newPipe(slots, pageSize int) *pipe {
	p := &pipe{ring: newRing(slots, pageSize), readers: 1, writers: 1}
	p.readable = sync.NewCond(&p.mu)
	p.writable = sync.NewCond(&p.mu)

	return p
}


// write returns when every byte of data is in the ring, or when no reader is
// left. While the ring is full it sleeps on writable.
func (p *pipe) write(data []byte) (int, error) {
	p.mu.Lock()
	defer p.mu.Unlock()

	written := 0

	for written < len(data) {
		if p.readers == 0 {
			return written, errBrokenPipe
		}

		n := p.ring.put(data[written:])
		written += n

		if n == 0 {
			p.writable.Wait()
			continue
		}

		p.readable.Broadcast()
	}

	return written, nil
}


// read returns as soon as it has moved at least one byte into buf. While the
// ring is empty and a writer exists it sleeps on readable. It returns 0 when
// the ring is empty and no writer is left: the end of the stream.
func (p *pipe) read(buf []byte) int {
	p.mu.Lock()
	defer p.mu.Unlock()

	for p.ring.empty() && p.writers > 0 {
		p.readable.Wait()
	}

	n := p.ring.take(buf)

	if n > 0 {
		p.writable.Broadcast()
	}

	return n
}


// closeWriter gives up a write end. A reader asleep on an empty ring wakes
// and finds that no writer is left.
func (p *pipe) closeWriter() {
	p.mu.Lock()
	defer p.mu.Unlock()

	p.writers--
	p.readable.Broadcast()
}

// closeReader gives up a read end. A writer asleep on a full ring wakes and
// finds that no reader is left.
func (p *pipe) closeReader() {
	p.mu.Lock()
	defer p.mu.Unlock()

	p.readers--
	p.writable.Broadcast()
}


// picture renders the ring as one cell per slot: the number of unread bytes
// in its page, or a dot when the slot is free.
func (r *ring) picture() string {
	cells := make([]string, len(r.pages))

	for i := range cells {
		cells[i] = "."
	}

	for counter := r.tail; counter != r.head; counter++ {
		cells[counter%uint(len(r.pages))] = fmt.Sprint(r.slot(counter).length)
	}

	return "[" + strings.Join(cells, " ") + "]"
}

// smallRing applies the calls of the trace to a ring of 4 pages of 8 bytes
// and prints the ring after each.
func smallRing() {
	r := newRing(4, 8)
	buf := make([]byte, 32)

	calls := []struct {
		op string
		n  int
	}{{"put", 12}, {"take", 3}, {"put", 30}, {"take", 5}, {"put", 30}}

	fmt.Println("ring of 4 pages of 8 bytes")

	for _, call := range calls {
		var done int

		switch call.op {
		case "put":
			done = r.put(buf[:call.n])
		case "take":
			done = r.take(buf[:call.n])
		}

		fmt.Printf("%-4s %2d: %2d bytes   head %d  tail %d   %s\n", call.op, call.n, done, r.head, r.tail, r.picture())
	}
}


// fill puts whole blocks into the ring until it accepts none and returns the
// number of bytes accepted.
func fill(r *ring, block []byte) int {
	total := 0

	for {
		n := r.put(block)

		if n == 0 {
			return total
		}

		total += n
	}
}

// kernelSize repeats the capacity measurement of pipefacts on a ring with
// the kernel's dimensions: 16 pages of 4096 bytes.
func kernelSize() {
	const pageSize = 4096

	r := newRing(16, pageSize)
	block := make([]byte, pageSize)

	steps := []struct {
		read   int
		report string
	}{
		{0, "new pipe: %d bytes accepted\n"},
		{1, "after reading 1 byte: %d bytes accepted\n"},
		{pageSize - 1, "after reading 4096 bytes in total: %d bytes accepted\n"},
	}

	for _, step := range steps {
		r.take(block[:step.read])
		fmt.Printf(step.report, fill(r, block))
	}
}


// transfer sends one megabyte through a pipe of the kernel's size. Byte i of
// the stream has the value i mod 256, which lets the reader verify the order.
func transfer() {
	const total = 1 << 20

	p := newPipe(16, 4096)

	go func() {
		defer p.closeWriter()

		chunk := make([]byte, 100_000)

		for sent := 0; sent < total; sent += len(chunk) {
			chunk = chunk[:min(len(chunk), total-sent)]

			for i := range chunk {
				chunk[i] = byte(sent + i)
			}

			if _, err := p.write(chunk); err != nil {
				return
			}
		}
	}()

	buf := make([]byte, 30_000)
	received, inOrder := 0, true

	for n := p.read(buf); n > 0; n = p.read(buf) {
		for i, b := range buf[:n] {
			inOrder = inOrder && b == byte(received+i)
		}

		received += n
	}

	fmt.Printf("1048576 bytes written: %d bytes read, in order: %v\n", received, inOrder)
	fmt.Printf("write end closed: read returns %d\n", p.read(buf))

	p.closeReader()

	_, err := p.write([]byte("hello"))
	fmt.Printf("read end closed: write fails with %v\n", err)
}


func main() {
	smallRing()
	fmt.Println()
	kernelSize()
	fmt.Println()
	transfer()
}

Listing 1.48 The complete handpipe, in Rust. code/rust/handpipe/src/main.rs

//! Builds a pipe by hand, with the same parts the kernel uses: a ring of
//! pages for the bytes, a lock, a place for a reader to sleep and a place for
//! a writer to sleep, and a count of the open ends.
//!
//! It traces a small ring call by call, repeats the capacity measurement of
//! pipefacts on a ring of the kernel's size, and sends 1048576 bytes from a
//! writer thread to a reader.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p handpipe
//! ```

use std::fmt;
use std::thread;

use parking_lot::{Condvar, Mutex};

/// One slot of the ring: a block of memory, and the position and length of
/// the bytes in it that have not been read yet.
struct Page {
    data: Vec<u8>,
    offset: usize,
    length: usize,
}

/// The storage of a pipe. `head` counts the pages ever put into use and
/// `tail` counts the pages ever emptied, so `head - tail` pages are in use,
/// and a counter's remainder after division by the number of slots is its
/// slot.
struct Ring {
    pages: Vec<Page>,
    head: usize,
    tail: usize,
}

impl Ring {
    fn new(slots: usize, page_size: usize) -> Ring {
        let pages = (0..slots)
            .map(|_| Page {
                data: vec![0; page_size],
                offset: 0,
                length: 0,
            })
            .collect();

        Ring {
            pages,
            head: 0,
            tail: 0,
        }
    }

    fn empty(&self) -> bool {
        self.head == self.tail
    }

    fn full(&self) -> bool {
        self.head - self.tail == self.pages.len()
    }

    /// The page that a counter refers to.
    fn slot(&mut self, counter: usize) -> &mut Page {
        let slots = self.pages.len();

        &mut self.pages[counter % slots]
    }
}

/// Copies as many bytes as both slices allow and returns the count.
fn copy(to: &mut [u8], from: &[u8]) -> usize {
    let n = to.len().min(from.len());
    to[..n].copy_from_slice(&from[..n]);

    n
}

impl Ring {
    /// Copies as much of `data` as the ring has room for and returns the
    /// number of bytes it copied. The bytes go first into the unused end of
    /// the newest page, then into free slots, one page each.
    fn put(&mut self, data: &[u8]) -> usize {
        let mut copied = 0;

        if !self.empty() {
            let newest = self.slot(self.head - 1);
            let end = newest.offset + newest.length;

            let n = copy(&mut newest.data[end..], data);
            newest.length += n;
            copied += n;
        }

        while copied < data.len() && !self.full() {
            let next = self.slot(self.head);
            next.offset = 0;
            next.length = copy(&mut next.data, &data[copied..]);

            copied += next.length;
            self.head += 1;
        }

        copied
    }
}

impl Ring {
    /// Moves up to `buf.len()` of the oldest bytes into `buf` and returns the
    /// number it moved. A slot becomes free when the last byte of its page
    /// has been taken.
    fn take(&mut self, buf: &mut [u8]) -> usize {
        let mut moved = 0;

        while moved < buf.len() && !self.empty() {
            let oldest = self.slot(self.tail);
            let unread = oldest.offset..oldest.offset + oldest.length;

            let n = copy(&mut buf[moved..], &oldest.data[unread]);
            oldest.offset += n;
            oldest.length -= n;
            moved += n;

            if oldest.length == 0 {
                self.tail += 1;
            }
        }

        moved
    }
}

/// What a write returns when no reader is left.
#[derive(Debug)]
struct BrokenPipe;

impl fmt::Display for BrokenPipe {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("EPIPE")
    }
}

/// What the lock guards: the ring and the count of the open ends.
struct State {
    ring: Ring,
    readers: usize,
    writers: usize,
}

/// A ring shared by a reader and a writer thread. `readable` is where a reader sleeps while
/// the ring is empty, and `writable` is where a writer sleeps while the ring
/// is full.
struct Pipe {
    state: Mutex<State>,
    readable: Condvar,
    writable: Condvar,
}

impl Pipe {
    fn new(slots: usize, page_size: usize) -> Pipe {
        let state = State {
            ring: Ring::new(slots, page_size),
            readers: 1,
            writers: 1,
        };

        Pipe {
            state: Mutex::new(state),
            readable: Condvar::new(),
            writable: Condvar::new(),
        }
    }
}

impl Pipe {
    /// Returns when every byte of `data` is in the ring, or when no reader is
    /// left. While the ring is full it sleeps on `writable`.
    fn write(&self, data: &[u8]) -> Result<usize, BrokenPipe> {
        let mut state = self.state.lock();
        let mut written = 0;

        while written < data.len() {
            if state.readers == 0 {
                return Err(BrokenPipe);
            }

            let n = state.ring.put(&data[written..]);
            written += n;

            if n == 0 {
                self.writable.wait(&mut state);
                continue;
            }

            self.readable.notify_all();
        }

        Ok(written)
    }
}

impl Pipe {
    /// Returns as soon as it has moved at least one byte into `buf`. While
    /// the ring is empty and a writer exists it sleeps on `readable`. It
    /// returns 0 when the ring is empty and no writer is left: the end of the
    /// stream.
    fn read(&self, buf: &mut [u8]) -> usize {
        let mut state = self.state.lock();

        self.readable
            .wait_while(&mut state, |state| state.ring.empty() && state.writers > 0);

        let n = state.ring.take(buf);

        if n > 0 {
            self.writable.notify_all();
        }

        n
    }
}

impl Pipe {
    /// Gives up a write end. A reader asleep on an empty ring wakes and finds
    /// that no writer is left.
    fn close_writer(&self) {
        self.state.lock().writers -= 1;
        self.readable.notify_all();
    }

    /// Gives up a read end. A writer asleep on a full ring wakes and finds
    /// that no reader is left.
    fn close_reader(&self) {
        self.state.lock().readers -= 1;
        self.writable.notify_all();
    }
}

impl Ring {
    /// Renders the ring as one cell per slot: the unread bytes in its page,
    /// or a dot for a free slot.
    fn picture(&self) -> String {
        let slots = self.pages.len();
        let mut cells = vec![".".to_string(); slots];

        for counter in self.tail..self.head {
            cells[counter % slots] = self.pages[counter % slots].length.to_string();
        }

        format!("[{}]", cells.join(" "))
    }
}

/// Applies the calls of the trace to a ring of 4 pages of 8 bytes and prints
/// the ring after each.
fn small_ring() {
    let mut ring = Ring::new(4, 8);
    let mut buf = [0u8; 32];

    let calls = [
        ("put", 12),
        ("take", 3),
        ("put", 30),
        ("take", 5),
        ("put", 30),
    ];

    println!("ring of 4 pages of 8 bytes");

    for (op, n) in calls {
        let done = match op {
            "put" => ring.put(&buf[..n]),
            _ => ring.take(&mut buf[..n]),
        };

        println!(
            "{op:<4} {n:2}: {done:2} bytes   head {}  tail {}   {}",
            ring.head,
            ring.tail,
            ring.picture()
        );
    }
}

/// Puts whole blocks into the ring until it accepts none and returns the
/// number of bytes accepted.
fn fill(ring: &mut Ring, block: &[u8]) -> usize {
    let mut total = 0;

    loop {
        let n = ring.put(block);

        if n == 0 {
            return total;
        }

        total += n;
    }
}

/// Repeats the capacity measurement of pipefacts on a ring with the kernel's
/// dimensions: 16 pages of 4096 bytes.
fn kernel_size() {
    const PAGE_SIZE: usize = 4096;

    let mut ring = Ring::new(16, PAGE_SIZE);
    let mut block = [0u8; PAGE_SIZE];

    let steps = [
        (0, "new pipe"),
        (1, "after reading 1 byte"),
        (PAGE_SIZE - 1, "after reading 4096 bytes in total"),
    ];

    for (count, label) in steps {
        ring.take(&mut block[..count]);

        let accepted = fill(&mut ring, &block);
        println!("{label}: {accepted} bytes accepted");
    }
}

/// Sends one megabyte through a pipe of the kernel's size. Byte `i` of the
/// stream has the value `i` mod 256, which lets the reader verify the order.
fn transfer() {
    const TOTAL: usize = 1 << 20;

    let pipe = Pipe::new(16, 4096);
    let mut buf = vec![0u8; 30_000];

    thread::scope(|scope| {
        scope.spawn(|| {
            let mut chunk = vec![0u8; 100_000];

            for sent in (0..TOTAL).step_by(chunk.len()) {
                chunk.truncate(TOTAL - sent);

                for (i, byte) in chunk.iter_mut().enumerate() {
                    *byte = (sent + i) as u8;
                }

                if pipe.write(&chunk).is_err() {
                    break;
                }
            }

            pipe.close_writer();
        });

        let (mut received, mut in_order) = (0, true);

        loop {
            let n = pipe.read(&mut buf);

            if n == 0 {
                break;
            }

            for (i, byte) in buf[..n].iter().enumerate() {
                in_order = in_order && *byte == (received + i) as u8;
            }

            received += n;
        }

        println!("1048576 bytes written: {received} bytes read, in order: {in_order}");
    });

    println!("write end closed: read returns {}", pipe.read(&mut buf));

    pipe.close_reader();

    if let Err(error) = pipe.write(b"hello") {
        println!("read end closed: write fails with {error}");
    }
}

fn main() {
    small_ring();
    println!();
    kernel_size();
    println!();
    transfer();
}

Listing 1.49 The complete pipeline, in Go. code/go/pipeline/main.go

// Command pipeline runs `a | b | c` the way a shell does: each command's
// standard output is the write end of a pipe, and the next command's standard
// input is the read end of the same pipe.
//
// It runs the pipelines listed in main and prints the last command's output and how each
// command terminated.
//
// Run it from code/go with:
//
//	go run ./pipeline
package main

import (
	"fmt"
	"io"
	"log"
	"os"
	"os/exec"
	"strings"
	"syscall"
)


// start runs the commands of a pipeline. Command i writes to pipe i and reads
// from pipe i-1; the first command reads /dev/null. It returns the commands
// and the read end of the last pipe.
func start(stages [][]string) ([]*exec.Cmd, *os.File, error) {
	var input *os.File
	commands := make([]*exec.Cmd, 0, len(stages))

	for _, stage := range stages {
		reader, writer, err := os.Pipe()

		if err != nil {
			return nil, nil, err
		}

		command := exec.Command(stage[0], stage[1:]...)
		command.Stdout = writer

		if input != nil {
			command.Stdin = input
		}

		err = command.Start()

		// The child holds its own copies of both ends. Closing the parent's
		// copies leaves the child's as the only ones.
		writer.Close()

		if input != nil {
			input.Close()
		}

		if err != nil {
			reader.Close()

			return nil, nil, fmt.Errorf("%s: %w", stage[0], err)
		}

		commands, input = append(commands, command), reader
	}

	return commands, input, nil
}


// ending waits for a command and reports how it terminated: with an exit
// status, or by a signal.
func ending(command *exec.Cmd) string {
	command.Wait()

	state := command.ProcessState

	if status, ok := state.Sys().(syscall.WaitStatus); ok && status.Signaled() {
		return fmt.Sprintf("killed by signal %d", status.Signal())
	}

	return fmt.Sprintf("exited with status %d", state.ExitCode())
}


// run starts a pipeline, reads the last command's output to the end of the
// stream, and prints the output and each command's termination.
func run(line string) error {
	var stages [][]string

	for _, stage := range strings.Split(line, " | ") {
		stages = append(stages, strings.Fields(stage))
	}

	commands, output, err := start(stages)

	if err != nil {
		return err
	}

	defer output.Close()

	text, err := io.ReadAll(output)

	if err != nil {
		return err
	}

	fmt.Println(line)
	fmt.Printf("  output: %s\n", strings.Join(strings.Fields(string(text)), " "))

	for i, command := range commands {
		fmt.Printf("  %s: %s\n", stages[i][0], ending(command))
	}

	return nil
}


func main() {
	for _, line := range []string{"seq 1 100000 | grep 7 | wc -l", "yes | head -n 3"} {
		if err := run(line); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 1.50 The complete pipeline, in Rust. code/rust/pipeline/src/main.rs

//! Runs `a | b | c` the way a shell does: each command's standard output is
//! the write end of a pipe, and the next command's standard input is the read
//! end of the same pipe.
//!
//! It runs the pipelines listed in main and prints the last command's output and how each
//! command terminated.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p pipeline
//! ```

use std::io::{self, PipeReader, Read};
use std::os::unix::process::ExitStatusExt;
use std::process::{Child, Command, Stdio};

/// Runs the commands of a pipeline. Command `i` writes to pipe `i` and reads
/// from pipe `i - 1`; the first command reads /dev/null. Returns the children
/// and the read end of the last pipe.
fn start(stages: &[Vec<&str>]) -> io::Result<(Vec<Child>, PipeReader)> {
    let mut input = None;
    let mut children = Vec::with_capacity(stages.len());

    for stage in stages {
        let (reader, writer) = io::pipe()?;
        let stdin = input.take().map_or_else(Stdio::null, Stdio::from);

        // The `Command` owns the parent's copies of both ends. It is dropped
        // when this statement ends, which closes them and leaves the child's
        // copies as the only ones.
        let child = Command::new(stage[0])
            .args(&stage[1..])
            .stdin(stdin)
            .stdout(writer)
            .spawn()?;

        children.push(child);
        input = Some(reader);
    }

    let output = input.ok_or(io::ErrorKind::InvalidInput)?;

    Ok((children, output))
}

/// Waits for a command and reports how it terminated: with an exit status,
/// or by a signal.
fn ending(child: &mut Child) -> io::Result<String> {
    let status = child.wait()?;

    let text = match (status.signal(), status.code()) {
        (Some(signal), _) => format!("killed by signal {signal}"),
        (None, Some(code)) => format!("exited with status {code}"),
        (None, None) => status.to_string(),
    };

    Ok(text)
}

/// Starts a pipeline, reads the last command's output to the end of the
/// stream, and prints the output and each command's termination.
fn run(line: &str) -> io::Result<()> {
    let stages: Vec<Vec<&str>> = line
        .split(" | ")
        .map(|stage| stage.split_whitespace().collect())
        .collect();

    let (mut children, mut output) = start(&stages)?;

    let mut text = String::new();
    output.read_to_string(&mut text)?;

    let words: Vec<&str> = text.split_whitespace().collect();

    println!("{line}");
    println!("  output: {}", words.join(" "));

    for (stage, child) in stages.iter().zip(&mut children) {
        println!("  {}: {}", stage[0], ending(child)?);
    }

    Ok(())
}

fn main() -> io::Result<()> {
    run("seq 1 100000 | grep 7 | wc -l")?;
    run("yes | head -n 3")
}

References

• pipe(7) and pipe(2), Linux manual pages: https://man7.org/linux/man-pages/man7/pipe.7.html
• fs/pipe.c and include/linux/pipe_fs_i.h, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/fs/pipe.c
• M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010, chapter 44.
• D. M. Ritchie, “The Evolution of the Unix Time-sharing System”, AT&T Bell Laboratories Technical Journal, 63(8), 1984.
• Go, package os/signal, the section on SIGPIPE: https://pkg.go.dev/os/signal

File Descriptors

Every read, write and close in a program takes a small integer as its first argument. That integer is a file descriptor. It is the only handle a process has on a file, a pipe, a terminal or a network connection, and the kernel translates it on every call into the object the process means.

In this chapter, we will follow that translation from the integer to the bytes on disk. We will meet the kernel structures on the way, one at a time and in the order a read visits them: the descriptor table of the process, the open file, the directory entry, and the inode. For each structure we will look at its definition in the kernel source, say what each field is for, and confirm its behaviour with an experiment. Then we will write fdfacts, which measures all of this on a running kernel, and fdtable, a model of the same structures that gives the same results.

Why a number

A program runs in user mode, in which the processor lets it touch only its own memory. The kernel runs in kernel mode and keeps its data in memory that user-mode code cannot read or write. A system call is the controlled way from one to the other: the program puts a call number and arguments in registers and executes an instruction that switches the processor to kernel mode at a fixed entry point.

The objects that describe an open file live in kernel memory. The kernel cannot hand a process a pointer to them, because the process could not use the pointer and must not be able to forge one. It keeps a table for each process and gives the process an index into that table. The process can pass any integer it likes. The kernel looks the integer up in the caller’s own table, so a process can only ever name the files that the kernel has put there.

By convention the first 3 entries have fixed roles: descriptor 0 is the standard input, 1 the standard output, and 2 the standard error. A program started from a terminal finds all 3 referring to that terminal.

One read, step by step

Suppose a process has opened a file and received descriptor 3, and now calls read(3, buf, 100). The kernel does the following.

1. It takes entry 3 of the calling process’s descriptor table. The entry is a pointer to a struct file, the kernel’s record of one open file. An empty entry ends the call with the error EBADF, “bad file descriptor”.
2. It reads the current position in the file from the struct file. This position is the offset: the number of bytes from the start of the file at which the next read or write takes place.
3. It calls the read function of whatever the file is. The struct file holds a table of functions, and a disk file, a pipe and a socket each supply their own.
4. For a disk file, that function finds the file’s inode, the kernel’s record of the file itself, and through it the cached pages of the file. The page cache is the subject of chapter 8.
5. It copies up to 100 bytes from those pages into buf, in the memory of the process.
6. It adds the number of bytes copied to the offset and returns that number.

The steps name 3 objects: the table entry, the open file, and the inode. The rest of the chapter takes them in that order. The question to keep in mind for each is which state it holds, because that decides what is shared when a descriptor is duplicated or inherited.

The descriptor table

Each process has a struct files_struct, which contains its descriptor table. The table itself is a struct fdtable. These are the definitions from include/linux/fdtable.h in Linux 5.15, with the fields that concern locking left out.

struct fdtable {
        unsigned int max_fds;          /* number of entries in fd[] */
        struct file **fd;              /* the table: one pointer per descriptor */
        unsigned long *close_on_exec;  /* bitmap: descriptors closed by exec */
        unsigned long *open_fds;       /* bitmap: descriptors in use */
        /* ... */
};

struct files_struct {
        atomic_t count;                /* processes and threads sharing this table */
        struct fdtable *fdt;           /* the current table */
        unsigned int next_fd;          /* where the search for a free number starts */
        struct file *fd_array[64];     /* a first table, large enough for most programs */
        /* ... */
};

The table gives rise to 2 rules that every Unix program depends on.

A new descriptor gets the lowest free number. The kernel searches open_fds for the first clear bit, starting at next_fd. After a process closes descriptor 0, the next open returns 0.

The number of descriptors is limited. Each process has a resource limit, RLIMIT_NOFILE, and an open that would need a number at or above it fails with EMFILE, “too many open files”. ulimit -n shows the limit in a shell. On this machine it is 1048576 for a process started from the terminal, and many systems set 1024. The kernel as a whole has a second limit, on the number of open files in all processes together. /proc/sys/fs/file-nr shows the current number first:

$ cat /proc/sys/fs/file-nr
17312	0	9223372036854775807

A server that forgets to close descriptors, a descriptor leak, eventually hits EMFILE and can accept no more connections, although memory and CPU are free.

The open file

The pointers in the table refer to struct file objects. The POSIX standard calls this object an open file description, and the name is worth learning because the manual pages use it. A struct file is created by each successful open, and also by pipe, socket and accept. This is the definition from include/linux/fs.h, reduced to the fields this chapter uses.

struct file {
        struct path f_path;                    /* the directory entry it was opened by */
        struct inode *f_inode;                 /* the file itself */
        const struct file_operations *f_op;    /* read, write, ... for this kind of file */
        atomic_long_t f_count;                 /* references to this struct file */
        unsigned int f_flags;                  /* O_APPEND, O_NONBLOCK, ... */
        fmode_t f_mode;                        /* opened for reading, writing, or both */
        loff_t f_pos;                          /* the offset */
        /* ... */
};

f_op is the mechanism behind the phrase “everything is a file”. The system call read contains no code for disks, pipes or sockets. It calls through f_op, and each kind of object installs its own functions when the struct file is created: ext4 for a file on disk, the pipe code of chapter 1 for a pipe, TCP for the socket of chapter 10. A program that reads descriptor 0 therefore works unchanged whether descriptor 0 is a terminal, a file or a pipe.

The important point for the rest of the chapter is where the offset lives. It is in the struct file, and neither in the table entry nor in the inode. Table entries that point to the same struct file share an offset. Separate struct file objects for the same file on disk each have their own.

Names, directory entries and inodes

A struct file refers to an inode. The inode is the file: its type, its owner, its permissions, its size, its timestamps, and the location of its data. It is identified by a number, the inode number, which is unique within a file system.

An inode contains no name. Names are kept in directories. A directory is itself a file whose contents are a list of directory entries, each a pair of a name and an inode number. Opening /tmp/x/report.txt means reading the directory /, finding the entry tmp and its inode, reading that directory, and so on to the last name. The kernel caches each step in a struct dentry, so that a path which has been used recently is resolved without reading directories from disk.

struct dentry {
        struct dentry *d_parent;       /* the directory that contains this entry */
        struct qstr d_name;            /* the name: a single component of a path */
        struct inode *d_inode;         /* the inode the name refers to */
        /* ... */
};

struct inode {
        umode_t i_mode;                /* type (file, directory, ...) and permissions */
        kuid_t i_uid;                  /* owner */
        kgid_t i_gid;                  /* group */
        unsigned long i_ino;           /* the inode number */
        unsigned int i_nlink;          /* directory entries that refer to this inode */
        loff_t i_size;                 /* size in bytes */
        struct timespec64 i_atime;     /* last access */
        struct timespec64 i_mtime;     /* last change of the contents */
        struct timespec64 i_ctime;     /* last change of the inode */
        blkcnt_t i_blocks;             /* disk space used, in 512-byte units */
        atomic_t i_count;              /* references from inside the kernel */
        const struct file_operations *i_fop;   /* copied to f_op at open */
        struct address_space *i_mapping;       /* the cached pages of the file */
        /* ... */
};

stat prints the fields of an inode, and ls -i prints inode numbers. Since the name is outside the inode, several directory entries can hold the same inode number. Such an entry is a hard link, created by ln or the link system call. On this machine:

$ echo hello > report.txt
$ ln report.txt copy.txt
$ ls -li
19927765 -rw-rw-r-- 2 jetson jetson 6 Oct 12 22:56 copy.txt
19927765 -rw-rw-r-- 2 jetson jetson 6 Oct 12 22:56 report.txt
$ stat report.txt
  File: report.txt
  Size: 6         	Blocks: 8          IO Block: 4096   regular file
Device: 10301h/66305d	Inode: 19927765    Links: 2
Access: (0664/-rw-rw-r--)  Uid: ( 1000/  jetson)   Gid: ( 1000/  jetson)

Both names show inode 19927765. The 2 after the permissions, Links: 2 in the output of stat, is i_nlink. The names are equal in every respect: neither is the original, and a change made through one is visible through the other, because there is a single inode and a single copy of the data.

Several everyday facts follow from this split.

• Renaming a file within a file system is fast at any size. rename changes a directory entry and does not touch the inode or the data.
• Permissions belong to the inode, so they are the same through every name.
• Removing a file is called unlink because it removes a link: a directory entry. What happens to the inode is the subject of the next section.

Figure 2.4 puts the structures together.

What is shared

We can now answer the question a programmer has to ask: when a process has 2 descriptors, what do they share? It depends on how the second descriptor came to exist.

The reason for sharing the offset through dup and fork is sequential output. When a shell runs (date; ls) > file, both commands inherit the same struct file, and ls continues at the offset where date stopped. If each had its own offset, the second command would overwrite the output of the first. We will measure exactly that below.

The close-on-exec flag is deliberately excluded from sharing. It tells the kernel to close the descriptor when the process executes a new program, and it is stored in the close_on_exec bitmap of the table, outside the struct file. A program can therefore keep a descriptor for itself and still pass a duplicate of it to a child.

Lifetimes

An object in the kernel is released when nothing refers to it any more, and each of the objects of this chapter keeps a count for that purpose.

A struct file is closed when the last descriptor for it is closed. close on a descriptor that was duplicated or inherited only lowers f_count. This is the rule that chapter 1 needed: a pipe reports the end of the stream when the last descriptor for its write end is gone, in whichever process that descriptor was.

An inode and its data are freed when it has no name and is not open. Both conditions are required. A file that is unlinked while a process has it open keeps its inode and its data until that process closes it.

The kernel shows such a file under /proc. Here a shell opens report.txt on descriptor 7 and then removes both names:

$ exec 7< report.txt
$ rm report.txt copy.txt
$ ls -l /proc/$$/fd/7
7 -> /tmp/x/report.txt (deleted)
$ cat <&7
hello

The consequences are practical. A program can create a temporary file, open it and unlink it at once, and the file disappears when the program ends, however it ends. And a large log file that is deleted while a server still writes to it continues to occupy the disk, which is the usual explanation when df reports a full disk and du cannot find the files.

fork and exec

fork gives the child a copy of the parent’s descriptor table. The copy is a new struct fdtable with the same pointers, and f_count of every open file is raised by 1. Parent and child then have independent tables, so either can close or reassign a descriptor without affecting the other, and they share every open file that both still hold.

exec replaces the program and keeps the table, with 1 exception: descriptors whose close-on-exec bit is set are closed. A program sets the bit with the O_CLOEXEC flag of open, or later with fcntl and F_SETFD. Without it, every descriptor a program has open is passed to every program it starts, including descriptors for files those programs have no business with. Go’s standard library and Rust’s standard library set the flag on every descriptor they create, and pass a descriptor to a child only when the program asks for it.

Chapter 3 covers fork and exec themselves.

Measuring descriptors

fdfacts checks the statements of this chapter on a running kernel. It uses golang.org/x/sys/unix in Go and the nix crate with the fs and resource features in Rust.

$ cargo new fdfacts
$ cargo add -p fdfacts nix --features fs,resource

numbers closes descriptor 0 and opens a file.

Listing 2.1 The imports of fdfacts, in Go. code/go/fdfacts/main.go

import (
	"errors"
	"fmt"
	"log"
	"os"
	"os/exec"
	"path/filepath"
	"strings"

	"golang.org/x/sys/unix"
)

Listing 2.2 The imports of fdfacts, in Rust. code/rust/fdfacts/src/main.rs

use std::fs::{self, File, OpenOptions};
use std::io::{self, Read};
use std::os::fd::{AsFd, AsRawFd};
use std::os::unix::fs::MetadataExt;
use std::path::Path;
use std::process::Command;

use nix::fcntl::{FcntlArg, FdFlag, OFlag, fcntl, open};
use nix::sys::resource::{Resource, getrlimit, setrlimit};
use nix::sys::stat::Mode;
use nix::unistd::{Whence, close, dup, lseek, write};

Listing 2.3 The number of a new descriptor, in Go. code/go/fdfacts/main.go

// numbers closes descriptor 0, the standard input, and opens a file. The
// kernel gives an open the lowest number that is free.
func numbers(path string) error {
	if err := unix.Close(0); err != nil {
		return err
	}

	fd, err := unix.Open(path, unix.O_RDONLY, 0)

	if err != nil {
		return err
	}

	fmt.Printf("descriptor 0 closed: the next open returns %d\n", fd)

	return nil
}

Listing 2.4 The number of a new descriptor, in Rust. code/rust/fdfacts/src/main.rs

/// Closes descriptor 0, the standard input, and opens a file. The kernel
/// gives an open the lowest number that is free.
fn numbers(path: &Path) -> io::Result<()> {
    close(0)?;

    let fd = open(path, OFlag::O_RDONLY, Mode::empty())?;
    println!(
        "descriptor 0 closed: the next open returns {}",
        fd.as_raw_fd()
    );

    Ok(())
}

limit lowers RLIMIT_NOFILE to 32 and opens a file until open fails. It reports the highest number it received and the error.

Listing 2.5 The limit on descriptors, in Go. code/go/fdfacts/main.go

// name returns the symbolic name of a system call error, such as EMFILE.
func name(err error) string {
	var errno unix.Errno

	if errors.As(err, &errno) {
		return unix.ErrnoName(errno)
	}

	return fmt.Sprint(err)
}

// limit lowers the limit on open descriptors to 32 and opens a file until
// the kernel refuses.
func limit(path string) error {
	var saved unix.Rlimit

	if err := unix.Getrlimit(unix.RLIMIT_NOFILE, &saved); err != nil {
		return err
	}

	lowered := saved
	lowered.Cur = 32

	if err := unix.Setrlimit(unix.RLIMIT_NOFILE, &lowered); err != nil {
		return err
	}

	defer unix.Setrlimit(unix.RLIMIT_NOFILE, &saved)

	var opened []int
	highest := -1

	for {
		fd, err := unix.Open(path, unix.O_RDONLY, 0)

		if err != nil {
			fmt.Printf("limit of 32 descriptors: the highest number obtained is %d, the next open fails with %s\n",
				highest, name(err))

			break
		}

		opened, highest = append(opened, fd), max(highest, fd)
	}

	for _, fd := range opened {
		unix.Close(fd)
	}

	return nil
}

Listing 2.6 The limit on descriptors, in Rust. code/rust/fdfacts/src/main.rs

/// Lowers the limit on open descriptors to 32 and opens a file until the
/// kernel refuses.
fn limit(path: &Path) -> io::Result<()> {
    let (soft, hard) = getrlimit(Resource::RLIMIT_NOFILE)?;

    setrlimit(Resource::RLIMIT_NOFILE, 32, hard)?;

    let mut opened = Vec::new();

    let error = loop {
        match open(path, OFlag::O_RDONLY, Mode::empty()) {
            Ok(fd) => opened.push(fd),
            Err(error) => break error,
        }
    };

    let highest = opened.iter().map(AsRawFd::as_raw_fd).fold(-1, i32::max);

    println!(
        "limit of 32 descriptors: the highest number obtained is {highest}, the next open fails with {error:?}"
    );

    drop(opened);
    setrlimit(Resource::RLIMIT_NOFILE, soft, hard)?;

    Ok(())
}

offsets opens a file, duplicates the descriptor, and opens the file again. It writes 5 bytes through the first descriptor and reads the offset through the other 2. lseek with a distance of 0 from the current position returns the offset without changing it.

Listing 2.7 The offset through a duplicate and through a second open, in Go. code/go/fdfacts/main.go

// offset returns the file offset that a descriptor refers to.
func offset(fd int) (int64, error) {
	return unix.Seek(fd, 0, unix.SEEK_CUR)
}

// offsets writes through one descriptor and reads the offset through a
// duplicate of it and through a second open of the same file.
func offsets(path string) error {
	first, err := unix.Open(path, unix.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(first)

	duplicate, err := unix.Dup(first)

	if err != nil {
		return err
	}

	defer unix.Close(duplicate)

	second, err := unix.Open(path, unix.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(second)

	if _, err := unix.Write(first, []byte("hello")); err != nil {
		return err
	}

	shared, err := offset(duplicate)

	if err != nil {
		return err
	}

	separate, err := offset(second)

	if err != nil {
		return err
	}

	fmt.Printf("after a write of 5 bytes: offset through the duplicate %d, through a second open %d\n",
		shared, separate)

	return nil
}

Listing 2.8 The offset through a duplicate and through a second open, in Rust. code/rust/fdfacts/src/main.rs

/// Returns the file offset that a descriptor refers to.
fn offset(fd: impl AsFd) -> nix::Result<i64> {
    lseek(fd, 0, Whence::SeekCur)
}

/// Writes through one descriptor and reads the offset through a duplicate of
/// it and through a second open of the same file.
fn offsets(path: &Path) -> io::Result<()> {
    let first = open(path, OFlag::O_WRONLY, Mode::empty())?;
    let duplicate = dup(&first)?;
    let second = open(path, OFlag::O_WRONLY, Mode::empty())?;

    write(&first, b"hello")?;

    let shared = offset(&duplicate)?;
    let separate = offset(&second)?;

    println!(
        "after a write of 5 bytes: offset through the duplicate {shared}, through a second open {separate}"
    );

    Ok(())
}

flags sets O_APPEND and close-on-exec on a duplicate and reads both through the original. F_GETFL and F_SETFL address f_flags in the struct file. F_GETFD and F_SETFD address the bitmap in the table.

Listing 2.9 Status flags and descriptor flags, in Go. code/go/fdfacts/main.go

// flags sets O_APPEND and close-on-exec through a duplicate and reads both
// through the original descriptor.
func flags(path string) error {
	first, err := unix.Open(path, unix.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(first)

	duplicate, err := unix.Dup(first)

	if err != nil {
		return err
	}

	defer unix.Close(duplicate)

	if _, err := unix.FcntlInt(uintptr(duplicate), unix.F_SETFL, unix.O_APPEND); err != nil {
		return err
	}

	if _, err := unix.FcntlInt(uintptr(duplicate), unix.F_SETFD, unix.FD_CLOEXEC); err != nil {
		return err
	}

	status, err := unix.FcntlInt(uintptr(first), unix.F_GETFL, 0)

	if err != nil {
		return err
	}

	descriptor, err := unix.FcntlInt(uintptr(first), unix.F_GETFD, 0)

	if err != nil {
		return err
	}

	fmt.Printf("set on the duplicate, read on the original: O_APPEND %v, close-on-exec %v\n",
		status&unix.O_APPEND != 0, descriptor&unix.FD_CLOEXEC != 0)

	return nil
}

Listing 2.10 Status flags and descriptor flags, in Rust. code/rust/fdfacts/src/main.rs

/// Sets `O_APPEND` and close-on-exec through a duplicate and reads both
/// through the original descriptor.
fn flags(path: &Path) -> io::Result<()> {
    let first = open(path, OFlag::O_WRONLY, Mode::empty())?;
    let duplicate = dup(&first)?;

    fcntl(&duplicate, FcntlArg::F_SETFL(OFlag::O_APPEND))?;
    fcntl(&duplicate, FcntlArg::F_SETFD(FdFlag::FD_CLOEXEC))?;

    let status = OFlag::from_bits_truncate(fcntl(&first, FcntlArg::F_GETFL)?);
    let descriptor = FdFlag::from_bits_truncate(fcntl(&first, FcntlArg::F_GETFD)?);

    println!(
        "set on the duplicate, read on the original: O_APPEND {}, close-on-exec {}",
        status.contains(OFlag::O_APPEND),
        descriptor.contains(FdFlag::FD_CLOEXEC)
    );

    Ok(())
}

names creates a file, gives it a second name with link, and compares the inode numbers behind the names.

Listing 2.11 A second name for a file, in Go. code/go/fdfacts/main.go

// inode returns the inode number and the link count of a file.
func inode(path string) (number uint64, links uint64, err error) {
	var status unix.Stat_t

	err = unix.Stat(path, &status)

	return status.Ino, uint64(status.Nlink), err
}

// names gives a file a second name and compares the inodes behind the names.
func names(path string) error {
	first, second := path+".first", path+".second"

	if err := os.WriteFile(first, []byte("hello"), 0o644); err != nil {
		return err
	}

	defer os.Remove(first)
	defer os.Remove(second)

	if err := unix.Link(first, second); err != nil {
		return err
	}

	a, _, err := inode(first)

	if err != nil {
		return err
	}

	b, links, err := inode(second)

	if err != nil {
		return err
	}

	fmt.Printf("a second name for the file: same inode %v, link count %d\n", a == b, links)

	return nil
}

Listing 2.12 A second name for a file, in Rust. code/rust/fdfacts/src/main.rs

/// Gives a file a second name and compares the inodes behind the names.
fn names(path: &Path) -> io::Result<()> {
    let (first, second) = (path.with_extension("first"), path.with_extension("second"));

    fs::write(&first, "hello")?;
    fs::hard_link(&first, &second)?;

    let (a, b) = (fs::metadata(&first)?, fs::metadata(&second)?);

    println!(
        "a second name for the file: same inode {}, link count {}",
        a.ino() == b.ino(),
        b.nlink()
    );

    fs::remove_file(&first)?;
    fs::remove_file(&second)
}

unlinked opens a file, removes its name, and then reads the link count and the contents through the descriptor.

Listing 2.13 A file removed while it is open, in Go. code/go/fdfacts/main.go

// unlinked opens a file, removes its only name, and reads through the
// descriptor.
func unlinked(path string) error {
	file := path + ".unlinked"

	if err := os.WriteFile(file, []byte("hello"), 0o644); err != nil {
		return err
	}

	fd, err := unix.Open(file, unix.O_RDONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(fd)

	if err := unix.Unlink(file); err != nil {
		return err
	}

	var status unix.Stat_t

	if err := unix.Fstat(fd, &status); err != nil {
		return err
	}

	buf := make([]byte, 16)
	n, err := unix.Read(fd, buf)

	if err != nil {
		return err
	}

	fmt.Printf("the name removed, the file still open: link count %d, read through the descriptor returns %q\n",
		status.Nlink, buf[:n])

	return nil
}

Listing 2.14 A file removed while it is open, in Rust. code/rust/fdfacts/src/main.rs

/// Opens a file, removes its only name, and reads through the descriptor.
fn unlinked(path: &Path) -> io::Result<()> {
    let name = path.with_extension("unlinked");

    fs::write(&name, "hello")?;

    let mut file = File::open(&name)?;

    fs::remove_file(&name)?;

    let links = file.metadata()?.nlink();
    let mut text = String::new();

    file.read_to_string(&mut text)?;

    println!(
        "the name removed, the file still open: link count {links}, read through the descriptor returns {text:?}"
    );

    Ok(())
}

redirections runs a shell command that writes out to descriptor 1 and then err to descriptor 2, with both descriptors on the same file. In the first run descriptor 2 is a duplicate of descriptor 1. In the other run the file is opened once for each.

Listing 2.15 Descriptors 1 and 2 on the same file, in Go. code/go/fdfacts/main.go

// redirect runs a command that writes "out" to descriptor 1 and then "err" to
// descriptor 2, with both descriptors on the same file, and returns the
// file's contents.
func redirect(path string, stdout, stderr *os.File) (string, error) {
	command := exec.Command("sh", "-c", "echo out; echo err >&2")
	command.Stdout, command.Stderr = stdout, stderr

	if err := command.Run(); err != nil {
		return "", err
	}

	text, err := os.ReadFile(path)

	return strings.Join(strings.Fields(string(text)), " "), err
}

// redirections compares `> file 2>&1`, where descriptor 2 is a duplicate of
// descriptor 1, with `> file 2> file`, where the file is opened once for each.
func redirections(path string) error {
	stdout, err := os.Create(path)

	if err != nil {
		return err
	}

	defer stdout.Close()

	text, err := redirect(path, stdout, stdout)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2>&1: the file holds %q\n", text)

	first, err := os.Create(path)

	if err != nil {
		return err
	}

	defer first.Close()

	second, err := os.OpenFile(path, os.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer second.Close()

	text, err = redirect(path, first, second)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2> file: the file holds %q\n", text)

	return nil
}

Listing 2.16 Descriptors 1 and 2 on the same file, in Rust. code/rust/fdfacts/src/main.rs

/// Runs a command that writes "out" to descriptor 1 and then "err" to
/// descriptor 2, with both descriptors on the same file, and returns the
/// file's contents.
fn redirect(path: &Path, stdout: File, stderr: File) -> io::Result<String> {
    Command::new("sh")
        .args(["-c", "echo out; echo err >&2"])
        .stdout(stdout)
        .stderr(stderr)
        .status()?;

    let text = fs::read_to_string(path)?;
    let words: Vec<&str> = text.split_whitespace().collect();

    Ok(words.join(" "))
}

/// Compares `> file 2>&1`, where descriptor 2 is a duplicate of descriptor 1,
/// with `> file 2> file`, where the file is opened once for each.
fn redirections(path: &Path) -> io::Result<()> {
    let stdout = File::create(path)?;
    let stderr = stdout.try_clone()?;

    let text = redirect(path, stdout, stderr)?;
    println!("> file 2>&1: the file holds {text:?}");

    let first = File::create(path)?;
    let second = OpenOptions::new().write(true).open(path)?;

    let text = redirect(path, first, second)?;
    println!("> file 2> file: the file holds {text:?}");

    Ok(())
}

descriptor 0 closed: the next open returns 0
limit of 32 descriptors: the highest number obtained is 31, the next open fails with EMFILE
after a write of 5 bytes: offset through the duplicate 5, through a second open 0
set on the duplicate, read on the original: O_APPEND true, close-on-exec false
a second name for the file: same inode true, link count 2
the name removed, the file still open: link count 0, read through the descriptor returns "hello"
> file 2>&1: the file holds "out err"
> file 2> file: the file holds "err"

Modelling the tables

fdtable implements the structures of this chapter in user space, with the same reference counts, and runs the experiments of fdfacts on them.

The Go version links the objects with pointers, as the kernel does. The Rust version keeps the inodes and the open file descriptions in tables owned by a Kernel value and refers to them by number, which avoids shared mutable ownership.

An inode has its contents and its 2 counts.

Listing 2.17 The imports of fdtable, in Go. code/go/fdtable/main.go

import (
	"errors"
	"fmt"
	"log"
	"strings"
)

Listing 2.18 The imports of fdtable, in Rust. code/rust/fdtable/src/main.rs

use std::collections::HashMap;
use std::fmt;

Listing 2.19 The inode, and the tables that hold inodes, in Go. code/go/fdtable/main.go

// The errors carry the names of the kernel's error codes for the same cases.
var (
	errBadDescriptor = errors.New("EBADF")
	errTooManyFiles  = errors.New("EMFILE")
	errNoEntry       = errors.New("ENOENT")
)

// An inode is a file: its number, its contents, and 2 reference counts.
// links counts the names that refer to it, and opens counts the open file
// descriptions that refer to it.
type inode struct {
	number int
	data   []byte
	links  int
	opens  int
}

// A filesystem holds the inodes and a single directory, which maps names to
// inodes.
type filesystem struct {
	names  map[string]*inode
	inodes map[int]*inode
	next   int
}

func newFilesystem() *filesystem {
	return &filesystem{names: map[string]*inode{}, inodes: map[int]*inode{}, next: 1}
}

// allocate returns a new inode with no name.
func (fs *filesystem) allocate() *inode {
	file := &inode{number: fs.next}
	fs.inodes[file.number] = file
	fs.next++

	return file
}

Listing 2.20 The inode, and the tables that hold inodes, in Rust. code/rust/fdtable/src/main.rs

/// The errors carry the names of the kernel's error codes for the same cases.
#[derive(Debug)]
enum Error {
    BadDescriptor,
    TooManyFiles,
    NoEntry,
}

impl fmt::Display for Error {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str(match self {
            Error::BadDescriptor => "EBADF",
            Error::TooManyFiles => "EMFILE",
            Error::NoEntry => "ENOENT",
        })
    }
}

/// A file: its contents and 2 reference counts. `links` counts the names
/// that refer to it, and `opens` counts the open file descriptions that refer
/// to it. Its number is its key in the kernel's table of inodes.
#[derive(Default)]
struct Inode {
    data: Vec<u8>,
    links: u32,
    opens: u32,
}

/// The kernel's tables: a single directory that maps names to inode numbers,
/// the inodes, and every open file description.
#[derive(Default)]
struct Kernel {
    names: HashMap<&'static str, u64>,
    inodes: HashMap<u64, Inode>,
    open_files: Vec<Description>,
    next: u64,
}

impl Kernel {
    /// Returns the number of a new inode with no name.
    fn allocate(&mut self) -> u64 {
        self.next += 1;
        self.inodes.insert(self.next, Inode::default());

        self.next
    }

    fn inode(&mut self, number: u64) -> Result<&mut Inode, Error> {
        self.inodes.get_mut(&number).ok_or(Error::NoEntry)
    }
}

create, link and unlink change the directory and the link count. release is the single place where an inode is freed, and it requires both counts to be 0.

Listing 2.21 Names: create, link, unlink, and release, in Go. code/go/fdtable/main.go

// create makes an empty file with one name.
func (fs *filesystem) create(name string) *inode {
	file := fs.allocate()
	file.links = 1
	fs.names[name] = file

	return file
}

// link adds a second name for an existing file.
func (fs *filesystem) link(existing, added string) error {
	file, ok := fs.names[existing]

	if !ok {
		return errNoEntry
	}

	file.links++
	fs.names[added] = file

	return nil
}

// unlink removes a name. The file itself is removed only when release finds
// no other reference to it.
func (fs *filesystem) unlink(name string) error {
	file, ok := fs.names[name]

	if !ok {
		return errNoEntry
	}

	delete(fs.names, name)
	file.links--
	fs.release(file)

	return nil
}

// release frees an inode that has no name and is not open.
func (fs *filesystem) release(file *inode) {
	if file.links == 0 && file.opens == 0 {
		delete(fs.inodes, file.number)
	}
}

Listing 2.22 Names: create, link, unlink, and release, in Rust. code/rust/fdtable/src/main.rs

impl Kernel {
    /// Makes an empty file with one name and returns its inode number.
    fn create(&mut self, name: &'static str) -> Result<u64, Error> {
        let number = self.allocate();

        self.inode(number)?.links = 1;
        self.names.insert(name, number);

        Ok(number)
    }

    /// Adds a second name for an existing file.
    fn link(&mut self, existing: &str, added: &'static str) -> Result<(), Error> {
        let number = *self.names.get(existing).ok_or(Error::NoEntry)?;

        self.inode(number)?.links += 1;
        self.names.insert(added, number);

        Ok(())
    }

    /// Removes a name. The file itself is removed only when `release` finds
    /// no other reference to it.
    fn unlink(&mut self, name: &str) -> Result<(), Error> {
        let number = self.names.remove(name).ok_or(Error::NoEntry)?;

        self.inode(number)?.links -= 1;
        self.release(number);

        Ok(())
    }

    /// Frees an inode that has no name and is not open.
    fn release(&mut self, number: u64) {
        if let Some(file) = self.inodes.get(&number)
            && file.links == 0
            && file.opens == 0
        {
            self.inodes.remove(&number);
        }
    }
}

The description corresponds to struct file, and the descriptor to a table entry together with its close-on-exec bit.

Listing 2.23 The description, the descriptor, and the process, in Go. code/go/fdtable/main.go

// A description is an open file description: the state created by one open.
// refs counts the descriptors, in all processes, that refer to it.
type description struct {
	file       *inode
	offset     int
	appendMode bool
	refs       int
}

// A descriptor is one entry of a process's table. The close-on-exec flag
// belongs to the entry.
type descriptor struct {
	open    *description
	cloexec bool
}

// A process is its descriptor table. The index of an entry is the file
// descriptor, a nil entry is a free number, and limit is the number of
// entries the process may use.
type process struct {
	fs    *filesystem
	table []*descriptor
	limit int
}

// newProcess returns a process with descriptors 0, 1 and 2 open on a
// terminal, which is a file without a name.
func newProcess(fs *filesystem) *process {
	p := &process{fs: fs, limit: 1024}
	terminal := &description{file: fs.allocate()}
	terminal.file.opens = 1

	for range 3 {
		p.install(terminal)
	}

	return p
}

Listing 2.24 The description, the descriptor, and the process, in Rust. code/rust/fdtable/src/main.rs

/// An open file description: the state created by one open. `file` is the
/// number of its inode, and `refs` counts the descriptors, in all processes,
/// that refer to it.
struct Description {
    file: u64,
    offset: usize,
    append_mode: bool,
    refs: u32,
}

/// One entry of a process's table. `open` is the index of a description in
/// the kernel's table. The close-on-exec flag belongs to the entry.
#[derive(Clone, Copy)]
struct Descriptor {
    open: usize,
    cloexec: bool,
}

/// A process is its descriptor table. The index of an entry is the file
/// descriptor, `None` is a free number, and `limit` is the number of entries
/// the process may use.
struct Process {
    table: Vec<Option<Descriptor>>,
    limit: usize,
}

impl Kernel {
    /// Adds a description for an inode and returns its index.
    fn describe(&mut self, file: u64) -> usize {
        self.open_files.push(Description {
            file,
            offset: 0,
            append_mode: false,
            refs: 0,
        });

        self.open_files.len() - 1
    }

    /// Returns a process with descriptors 0, 1 and 2 open on a terminal,
    /// which is a file without a name.
    fn process(&mut self) -> Result<Process, Error> {
        let mut process = Process {
            table: Vec::new(),
            limit: 1024,
        };

        let terminal = self.allocate();
        let open = self.describe(terminal);

        self.inode(terminal)?.opens = 1;

        for _ in 0..3 {
            self.install(&mut process, open)?;
        }

        Ok(process)
    }
}

install implements both rules of the descriptor table: the lowest free number and the limit. open creates a description, and dup installs an existing one, which is the whole difference between them.

Listing 2.25 Creating descriptors, in Go. code/go/fdtable/main.go

// install puts a description in the lowest free entry, counts the new
// reference, and returns the number of the entry.
func (p *process) install(open *description) (int, error) {
	fd := 0

	for fd < len(p.table) && p.table[fd] != nil {
		fd++
	}

	if fd >= p.limit {
		return 0, errTooManyFiles
	}

	if fd == len(p.table) {
		p.table = append(p.table, nil)
	}

	open.refs++
	p.table[fd] = &descriptor{open: open}

	return fd, nil
}

// lookup returns the entry of a descriptor.
func (p *process) lookup(fd int) (*descriptor, error) {
	if fd < 0 || fd >= len(p.table) || p.table[fd] == nil {
		return nil, errBadDescriptor
	}

	return p.table[fd], nil
}

// open finds the inode behind a name and creates a description for it, with
// offset 0.
func (p *process) open(name string) (int, error) {
	file, ok := p.fs.names[name]

	if !ok {
		return 0, errNoEntry
	}

	fd, err := p.install(&description{file: file})

	if err == nil {
		file.opens++
	}

	return fd, err
}

// dup makes a second descriptor for an existing description.
func (p *process) dup(fd int) (int, error) {
	entry, err := p.lookup(fd)

	if err != nil {
		return 0, err
	}

	return p.install(entry.open)
}

Listing 2.26 Creating descriptors, in Rust. code/rust/fdtable/src/main.rs

impl Kernel {
    /// Puts a description in the lowest free entry, counts the new reference,
    /// and returns the number of the entry.
    fn install(&mut self, process: &mut Process, open: usize) -> Result<usize, Error> {
        let fd = process
            .table
            .iter()
            .take_while(|entry| entry.is_some())
            .count();

        if fd >= process.limit {
            return Err(Error::TooManyFiles);
        }

        if fd == process.table.len() {
            process.table.push(None);
        }

        self.open_files[open].refs += 1;
        process.table[fd] = Some(Descriptor {
            open,
            cloexec: false,
        });

        Ok(fd)
    }

    /// Finds the inode behind a name and creates a description for it, with
    /// offset 0.
    fn open(&mut self, process: &mut Process, name: &str) -> Result<usize, Error> {
        let file = *self.names.get(name).ok_or(Error::NoEntry)?;
        let open = self.describe(file);
        let fd = self.install(process, open)?;

        self.inode(file)?.opens += 1;

        Ok(fd)
    }

    /// Makes a second descriptor for an existing description.
    fn dup(&mut self, process: &mut Process, fd: usize) -> Result<usize, Error> {
        let entry = process.lookup(fd)?;

        self.install(process, entry.open)
    }
}

impl Process {
    /// Returns the entry of a descriptor.
    fn lookup(&self, fd: usize) -> Result<Descriptor, Error> {
        self.table
            .get(fd)
            .copied()
            .flatten()
            .ok_or(Error::BadDescriptor)
    }
}

close lowers the count of the description and, at 0, the open count of the inode. dup2 closes its target first, which is why redirecting descriptor 1 releases whatever descriptor 1 was before.

Listing 2.27 Closing a descriptor, and dup2, in Go. code/go/fdtable/main.go

// close frees an entry and drops its reference to the description. The last
// reference closes the description, which drops its reference to the inode.
func (p *process) close(fd int) error {
	entry, err := p.lookup(fd)

	if err != nil {
		return err
	}

	p.table[fd] = nil
	entry.open.refs--

	if entry.open.refs == 0 {
		entry.open.file.opens--
		p.fs.release(entry.open.file)
	}

	return nil
}

// dup2 makes descriptor to refer to the description of descriptor from,
// closing whatever to referred to before.
func (p *process) dup2(from, to int) error {
	entry, err := p.lookup(from)

	if err != nil {
		return err
	}

	if from == to {
		return nil
	}

	if _, err := p.lookup(to); err == nil {
		p.close(to)
	}

	for len(p.table) <= to {
		p.table = append(p.table, nil)
	}

	entry.open.refs++
	p.table[to] = &descriptor{open: entry.open}

	return nil
}

Listing 2.28 Closing a descriptor, and dup2, in Rust. code/rust/fdtable/src/main.rs

impl Kernel {
    /// Frees an entry and drops its reference to the description. The last
    /// reference closes the description, which drops its reference to the
    /// inode.
    fn close(&mut self, process: &mut Process, fd: usize) -> Result<(), Error> {
        let entry = process.lookup(fd)?;
        let open = &mut self.open_files[entry.open];

        process.table[fd] = None;
        open.refs -= 1;

        if open.refs == 0 {
            let file = open.file;

            self.inode(file)?.opens -= 1;
            self.release(file);
        }

        Ok(())
    }

    /// Makes descriptor `to` refer to the description of descriptor `from`,
    /// closing whatever `to` referred to before.
    fn dup2(&mut self, process: &mut Process, from: usize, to: usize) -> Result<(), Error> {
        let entry = process.lookup(from)?;

        if from == to {
            return Ok(());
        }

        if process.lookup(to).is_ok() {
            self.close(process, to)?;
        }

        if process.table.len() <= to {
            process.table.resize(to + 1, None);
        }

        self.open_files[entry.open].refs += 1;
        process.table[to] = Some(Descriptor {
            open: entry.open,
            cloexec: false,
        });

        Ok(())
    }
}

read and write take the offset from the description. With appendMode set, write first moves the offset to the end of the file. The kernel performs that move and the write as a single atomic step, so concurrent appends do not overwrite each other.

Listing 2.29 Reading and writing through a descriptor, in Go. code/go/fdtable/main.go

// write copies data into the file at the description's offset and advances
// the offset. In append mode the offset first moves to the end of the file.
func (p *process) write(fd int, data string) error {
	entry, err := p.lookup(fd)

	if err != nil {
		return err
	}

	open, file := entry.open, entry.open.file

	if open.appendMode {
		open.offset = len(file.data)
	}

	end := open.offset + len(data)

	if end > len(file.data) {
		file.data = append(file.data, make([]byte, end-len(file.data))...)
	}

	copy(file.data[open.offset:], data)
	open.offset = end

	return nil
}

// read returns up to n bytes from the description's offset and advances the
// offset.
func (p *process) read(fd, n int) (string, error) {
	entry, err := p.lookup(fd)

	if err != nil {
		return "", err
	}

	open := entry.open
	end := min(open.offset+n, len(open.file.data))
	data := string(open.file.data[open.offset:end])
	open.offset = end

	return data, nil
}

Listing 2.30 Reading and writing through a descriptor, in Rust. code/rust/fdtable/src/main.rs

impl Kernel {
    /// Copies `data` into the file at the description's offset and advances
    /// the offset. In append mode the offset first moves to the end of the
    /// file.
    fn write(&mut self, process: &Process, fd: usize, data: &str) -> Result<(), Error> {
        let open = &mut self.open_files[process.lookup(fd)?.open];
        let file = self.inodes.get_mut(&open.file).ok_or(Error::NoEntry)?;

        if open.append_mode {
            open.offset = file.data.len();
        }

        let end = open.offset + data.len();

        if end > file.data.len() {
            file.data.resize(end, 0);
        }

        file.data[open.offset..end].copy_from_slice(data.as_bytes());
        open.offset = end;

        Ok(())
    }

    /// Returns up to `n` bytes from the description's offset and advances the
    /// offset.
    fn read(&mut self, process: &Process, fd: usize, n: usize) -> Result<String, Error> {
        let open = &mut self.open_files[process.lookup(fd)?.open];
        let file = self.inodes.get(&open.file).ok_or(Error::NoEntry)?;

        let end = (open.offset + n).min(file.data.len());
        let data = String::from_utf8_lossy(&file.data[open.offset..end]).into_owned();

        open.offset = end;

        Ok(data)
    }
}

fork copies the table and raises the count of every description. exec applies the close-on-exec flag.

Listing 2.31 Inheriting descriptors, in Go. code/go/fdtable/main.go

// fork returns a child whose table has a copy of every entry. The copies
// refer to the parent's descriptions, and each copy is counted.
func (p *process) fork() *process {
	child := &process{fs: p.fs, limit: p.limit, table: make([]*descriptor, len(p.table))}

	for fd, entry := range p.table {
		if entry != nil {
			entry.open.refs++
			child.table[fd] = &descriptor{open: entry.open, cloexec: entry.cloexec}
		}
	}

	return child
}

// exec closes every descriptor that has the close-on-exec flag.
func (p *process) exec() {
	for fd, entry := range p.table {
		if entry != nil && entry.cloexec {
			p.close(fd)
		}
	}
}

Listing 2.32 Inheriting descriptors, in Rust. code/rust/fdtable/src/main.rs

impl Kernel {
    /// Returns a child whose table has a copy of every entry. The copies
    /// refer to the parent's descriptions, and each copy is counted.
    fn fork(&mut self, parent: &Process) -> Process {
        for entry in parent.table.iter().flatten() {
            self.open_files[entry.open].refs += 1;
        }

        Process {
            table: parent.table.clone(),
            limit: parent.limit,
        }
    }

    /// Closes every descriptor that has the close-on-exec flag.
    fn exec(&mut self, process: &mut Process) -> Result<(), Error> {
        for fd in 0..process.table.len() {
            if process.table[fd].is_some_and(|entry| entry.cloexec) {
                self.close(process, fd)?;
            }
        }

        Ok(())
    }
}

The experiments repeat fdfacts on the model.

Listing 2.33 Numbers, the limit, offsets and flags on the model, in Go. code/go/fdtable/main.go

func numbers(fs *filesystem) error {
	p := newProcess(fs)

	if err := p.close(0); err != nil {
		return err
	}

	fd, err := p.open("file")
	fmt.Printf("descriptor 0 closed: the next open returns %d\n", fd)

	return err
}

func limit(fs *filesystem) error {
	p := newProcess(fs)
	p.limit = 32
	highest := -1

	for {
		fd, err := p.open("file")

		if err != nil {
			fmt.Printf("limit of 32 descriptors: the highest number obtained is %d, the next open fails with %v\n", highest, err)

			return nil
		}

		highest = max(highest, fd)
	}
}

func offsets(fs *filesystem) error {
	p := newProcess(fs)
	first, err := p.open("file")

	if err != nil {
		return err
	}

	duplicate, err := p.dup(first)

	if err != nil {
		return err
	}

	second, err := p.open("file")

	if err != nil {
		return err
	}

	if err := p.write(first, "hello"); err != nil {
		return err
	}

	fmt.Printf("after a write of 5 bytes: offset through the duplicate %d, through a second open %d\n",
		p.table[duplicate].open.offset, p.table[second].open.offset)

	return nil
}

func flags(fs *filesystem) error {
	p := newProcess(fs)
	first, err := p.open("file")

	if err != nil {
		return err
	}

	duplicate, err := p.dup(first)

	if err != nil {
		return err
	}

	p.table[duplicate].open.appendMode = true
	p.table[duplicate].cloexec = true

	fmt.Printf("set on the duplicate, read on the original: O_APPEND %v, close-on-exec %v\n",
		p.table[first].open.appendMode, p.table[first].cloexec)

	return nil
}

Listing 2.34 Numbers, the limit, offsets and flags on the model, in Rust. code/rust/fdtable/src/main.rs

fn numbers(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;

    kernel.close(&mut process, 0)?;

    let fd = kernel.open(&mut process, "file")?;
    println!("descriptor 0 closed: the next open returns {fd}");

    Ok(())
}

fn limit(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;
    let mut highest = 0;

    process.limit = 32;

    let error = loop {
        match kernel.open(&mut process, "file") {
            Ok(fd) => highest = highest.max(fd),
            Err(error) => break error,
        }
    };

    println!(
        "limit of 32 descriptors: the highest number obtained is {highest}, the next open fails with {error}"
    );

    Ok(())
}

fn offsets(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;

    let first = kernel.open(&mut process, "file")?;
    let duplicate = kernel.dup(&mut process, first)?;
    let second = kernel.open(&mut process, "file")?;

    kernel.write(&process, first, "hello")?;

    let shared = kernel.open_files[process.lookup(duplicate)?.open].offset;
    let separate = kernel.open_files[process.lookup(second)?.open].offset;

    println!(
        "after a write of 5 bytes: offset through the duplicate {shared}, through a second open {separate}"
    );

    Ok(())
}

fn flags(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;

    let first = kernel.open(&mut process, "file")?;
    let duplicate = kernel.dup(&mut process, first)?;

    kernel.open_files[process.lookup(duplicate)?.open].append_mode = true;

    if let Some(entry) = &mut process.table[duplicate] {
        entry.cloexec = true;
    }

    let original = process.lookup(first)?;

    println!(
        "set on the duplicate, read on the original: O_APPEND {}, close-on-exec {}",
        kernel.open_files[original.open].append_mode, original.cloexec
    );

    Ok(())
}

Listing 2.35 Names and the lifetime of an inode on the model, in Go. code/go/fdtable/main.go

func names(fs *filesystem) error {
	fs.create("first")

	if err := fs.link("first", "second"); err != nil {
		return err
	}

	a, b := fs.names["first"], fs.names["second"]
	fmt.Printf("a second name for the file: same inode %v, link count %d\n", a == b, b.links)

	return nil
}

func unlinked(fs *filesystem) error {
	p := newProcess(fs)
	file := fs.create("unlinked")
	file.data = []byte("hello")

	fd, err := p.open("unlinked")

	if err != nil {
		return err
	}

	if err := fs.unlink("unlinked"); err != nil {
		return err
	}

	data, err := p.read(fd, 16)

	if err != nil {
		return err
	}

	fmt.Printf("the name removed, the file still open: link count %d, read through the descriptor returns %q\n",
		file.links, data)

	return p.close(fd)
}

Listing 2.36 Names and the lifetime of an inode on the model, in Rust. code/rust/fdtable/src/main.rs

fn names(kernel: &mut Kernel) -> Result<(), Error> {
    kernel.create("first")?;
    kernel.link("first", "second")?;

    let (a, b) = (kernel.names["first"], kernel.names["second"]);

    println!(
        "a second name for the file: same inode {}, link count {}",
        a == b,
        kernel.inode(b)?.links
    );

    Ok(())
}

fn unlinked(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;
    let file = kernel.create("unlinked")?;

    kernel.inode(file)?.data = b"hello".to_vec();

    let fd = kernel.open(&mut process, "unlinked")?;

    kernel.unlink("unlinked")?;

    let links = kernel.inode(file)?.links;
    let data = kernel.read(&process, fd, 16)?;

    println!(
        "the name removed, the file still open: link count {links}, read through the descriptor returns {data:?}"
    );

    kernel.close(&mut process, fd)
}

redirect does what a shell does between fork and exec: it rearranges the child’s descriptors 1 and 2 with dup2. The parent’s table is unchanged.

Listing 2.37 Redirection on the model, in Go. code/go/fdtable/main.go

// redirect forks a child, points its descriptors 1 and 2 at the given
// descriptors, and has it write "out" and then "err". It returns the file's
// contents.
func redirect(shell *process, file *inode, stdout, stderr int) (string, error) {
	child := shell.fork()

	if err := child.dup2(stdout, 1); err != nil {
		return "", err
	}

	if err := child.dup2(stderr, 2); err != nil {
		return "", err
	}

	child.exec()

	if err := child.write(1, "out\n"); err != nil {
		return "", err
	}

	if err := child.write(2, "err\n"); err != nil {
		return "", err
	}

	return strings.Join(strings.Fields(string(file.data)), " "), nil
}

func redirections(fs *filesystem) error {
	shell := newProcess(fs)
	file := fs.create("output")
	stdout, err := shell.open("output")

	if err != nil {
		return err
	}

	text, err := redirect(shell, file, stdout, stdout)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2>&1: the file holds %q\n", text)

	file.data = nil
	first, err := shell.open("output")

	if err != nil {
		return err
	}

	second, err := shell.open("output")

	if err != nil {
		return err
	}

	text, err = redirect(shell, file, first, second)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2> file: the file holds %q\n", text)

	return nil
}

Listing 2.38 Redirection on the model, in Rust. code/rust/fdtable/src/main.rs

/// Forks a child, points its descriptors 1 and 2 at the given descriptors,
/// and has it write "out" and then "err". Returns the file's contents.
fn redirect(
    kernel: &mut Kernel,
    shell: &Process,
    file: u64,
    stdout: usize,
    stderr: usize,
) -> Result<String, Error> {
    let mut child = kernel.fork(shell);

    kernel.dup2(&mut child, stdout, 1)?;
    kernel.dup2(&mut child, stderr, 2)?;
    kernel.exec(&mut child)?;

    kernel.write(&child, 1, "out\n")?;
    kernel.write(&child, 2, "err\n")?;

    let text = String::from_utf8_lossy(&kernel.inode(file)?.data).into_owned();
    let words: Vec<&str> = text.split_whitespace().collect();

    Ok(words.join(" "))
}

fn redirections(kernel: &mut Kernel) -> Result<(), Error> {
    let mut shell = kernel.process()?;
    let file = kernel.create("output")?;
    let stdout = kernel.open(&mut shell, "output")?;

    let text = redirect(kernel, &shell, file, stdout, stdout)?;
    println!("> file 2>&1: the file holds {text:?}");

    kernel.inode(file)?.data.clear();

    let first = kernel.open(&mut shell, "output")?;
    let second = kernel.open(&mut shell, "output")?;

    let text = redirect(kernel, &shell, file, first, second)?;
    println!("> file 2> file: the file holds {text:?}");

    Ok(())
}

descriptor 0 closed: the next open returns 0
limit of 32 descriptors: the highest number obtained is 31, the next open fails with EMFILE
after a write of 5 bytes: offset through the duplicate 5, through a second open 0
set on the duplicate, read on the original: O_APPEND true, close-on-exec false
a second name for the file: same inode true, link count 2
the name removed, the file still open: link count 0, read through the descriptor returns "hello"
> file 2>&1: the file holds "out err"
> file 2> file: the file holds "err"

The output is identical to the output of fdfacts, line for line.

Redirection

A shell applies redirections from left to right, each as a dup2 in the child before exec.

• > file opens the file and makes descriptor 1 refer to the new struct file.
• 2>&1 makes descriptor 2 refer to the struct file that descriptor 1 refers to at that moment.

The order of redirections therefore changes the result. In > file 2>&1, descriptor 1 is on the file when descriptor 2 is duplicated from it, and both write to the file. In 2>&1 > file, descriptor 2 is duplicated first, while descriptor 1 is still the terminal. Descriptor 1 then moves to the file, and errors continue to go to the terminal.

>> file 2>> file is safe with separate struct file objects. Both are opened with O_APPEND, so each write goes to the current end of the file regardless of the offset.

Chapter 4 builds a shell that performs these redirections.

Inspecting a running process

The kernel publishes the descriptor table of every process under /proc, which makes the structures of this chapter visible on a live system.

For descriptor 7 of the shell above, after the names were removed:

$ cat /proc/$$/fdinfo/7
pos:	0
flags:	0400000
mnt_id:	28
ino:	19927765

pos is f_pos. flags is f_flags in octal: the last digit 0 is O_RDONLY, and 0400000 is O_LARGEFILE, which the kernel sets on every open on a 64-bit system. ino is the inode number, the same 19927765 that ls -li printed while the file had names.

The complete programs

Listing 2.39 The complete fdfacts, in Go. code/go/fdfacts/main.go

// Command fdfacts measures how the kernel hands out file descriptors and what
// a pair of descriptors shares: the number an open returns, the file offset, the
// status flags, and the close-on-exec flag. It ends with the redirection
// `> file 2>&1`, done once with a shared open file description and once with
// separate ones.
//
// Run it from code/go with:
//
//	go run ./fdfacts
package main

import (
	"errors"
	"fmt"
	"log"
	"os"
	"os/exec"
	"path/filepath"
	"strings"

	"golang.org/x/sys/unix"
)


// numbers closes descriptor 0, the standard input, and opens a file. The
// kernel gives an open the lowest number that is free.
func numbers(path string) error {
	if err := unix.Close(0); err != nil {
		return err
	}

	fd, err := unix.Open(path, unix.O_RDONLY, 0)

	if err != nil {
		return err
	}

	fmt.Printf("descriptor 0 closed: the next open returns %d\n", fd)

	return nil
}


// name returns the symbolic name of a system call error, such as EMFILE.
func name(err error) string {
	var errno unix.Errno

	if errors.As(err, &errno) {
		return unix.ErrnoName(errno)
	}

	return fmt.Sprint(err)
}

// limit lowers the limit on open descriptors to 32 and opens a file until
// the kernel refuses.
func limit(path string) error {
	var saved unix.Rlimit

	if err := unix.Getrlimit(unix.RLIMIT_NOFILE, &saved); err != nil {
		return err
	}

	lowered := saved
	lowered.Cur = 32

	if err := unix.Setrlimit(unix.RLIMIT_NOFILE, &lowered); err != nil {
		return err
	}

	defer unix.Setrlimit(unix.RLIMIT_NOFILE, &saved)

	var opened []int
	highest := -1

	for {
		fd, err := unix.Open(path, unix.O_RDONLY, 0)

		if err != nil {
			fmt.Printf("limit of 32 descriptors: the highest number obtained is %d, the next open fails with %s\n",
				highest, name(err))

			break
		}

		opened, highest = append(opened, fd), max(highest, fd)
	}

	for _, fd := range opened {
		unix.Close(fd)
	}

	return nil
}


// offset returns the file offset that a descriptor refers to.
func offset(fd int) (int64, error) {
	return unix.Seek(fd, 0, unix.SEEK_CUR)
}

// offsets writes through one descriptor and reads the offset through a
// duplicate of it and through a second open of the same file.
func offsets(path string) error {
	first, err := unix.Open(path, unix.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(first)

	duplicate, err := unix.Dup(first)

	if err != nil {
		return err
	}

	defer unix.Close(duplicate)

	second, err := unix.Open(path, unix.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(second)

	if _, err := unix.Write(first, []byte("hello")); err != nil {
		return err
	}

	shared, err := offset(duplicate)

	if err != nil {
		return err
	}

	separate, err := offset(second)

	if err != nil {
		return err
	}

	fmt.Printf("after a write of 5 bytes: offset through the duplicate %d, through a second open %d\n",
		shared, separate)

	return nil
}


// flags sets O_APPEND and close-on-exec through a duplicate and reads both
// through the original descriptor.
func flags(path string) error {
	first, err := unix.Open(path, unix.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(first)

	duplicate, err := unix.Dup(first)

	if err != nil {
		return err
	}

	defer unix.Close(duplicate)

	if _, err := unix.FcntlInt(uintptr(duplicate), unix.F_SETFL, unix.O_APPEND); err != nil {
		return err
	}

	if _, err := unix.FcntlInt(uintptr(duplicate), unix.F_SETFD, unix.FD_CLOEXEC); err != nil {
		return err
	}

	status, err := unix.FcntlInt(uintptr(first), unix.F_GETFL, 0)

	if err != nil {
		return err
	}

	descriptor, err := unix.FcntlInt(uintptr(first), unix.F_GETFD, 0)

	if err != nil {
		return err
	}

	fmt.Printf("set on the duplicate, read on the original: O_APPEND %v, close-on-exec %v\n",
		status&unix.O_APPEND != 0, descriptor&unix.FD_CLOEXEC != 0)

	return nil
}


// inode returns the inode number and the link count of a file.
func inode(path string) (number uint64, links uint64, err error) {
	var status unix.Stat_t

	err = unix.Stat(path, &status)

	return status.Ino, uint64(status.Nlink), err
}

// names gives a file a second name and compares the inodes behind the names.
func names(path string) error {
	first, second := path+".first", path+".second"

	if err := os.WriteFile(first, []byte("hello"), 0o644); err != nil {
		return err
	}

	defer os.Remove(first)
	defer os.Remove(second)

	if err := unix.Link(first, second); err != nil {
		return err
	}

	a, _, err := inode(first)

	if err != nil {
		return err
	}

	b, links, err := inode(second)

	if err != nil {
		return err
	}

	fmt.Printf("a second name for the file: same inode %v, link count %d\n", a == b, links)

	return nil
}


// unlinked opens a file, removes its only name, and reads through the
// descriptor.
func unlinked(path string) error {
	file := path + ".unlinked"

	if err := os.WriteFile(file, []byte("hello"), 0o644); err != nil {
		return err
	}

	fd, err := unix.Open(file, unix.O_RDONLY, 0)

	if err != nil {
		return err
	}

	defer unix.Close(fd)

	if err := unix.Unlink(file); err != nil {
		return err
	}

	var status unix.Stat_t

	if err := unix.Fstat(fd, &status); err != nil {
		return err
	}

	buf := make([]byte, 16)
	n, err := unix.Read(fd, buf)

	if err != nil {
		return err
	}

	fmt.Printf("the name removed, the file still open: link count %d, read through the descriptor returns %q\n",
		status.Nlink, buf[:n])

	return nil
}


// redirect runs a command that writes "out" to descriptor 1 and then "err" to
// descriptor 2, with both descriptors on the same file, and returns the
// file's contents.
func redirect(path string, stdout, stderr *os.File) (string, error) {
	command := exec.Command("sh", "-c", "echo out; echo err >&2")
	command.Stdout, command.Stderr = stdout, stderr

	if err := command.Run(); err != nil {
		return "", err
	}

	text, err := os.ReadFile(path)

	return strings.Join(strings.Fields(string(text)), " "), err
}

// redirections compares `> file 2>&1`, where descriptor 2 is a duplicate of
// descriptor 1, with `> file 2> file`, where the file is opened once for each.
func redirections(path string) error {
	stdout, err := os.Create(path)

	if err != nil {
		return err
	}

	defer stdout.Close()

	text, err := redirect(path, stdout, stdout)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2>&1: the file holds %q\n", text)

	first, err := os.Create(path)

	if err != nil {
		return err
	}

	defer first.Close()

	second, err := os.OpenFile(path, os.O_WRONLY, 0)

	if err != nil {
		return err
	}

	defer second.Close()

	text, err = redirect(path, first, second)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2> file: the file holds %q\n", text)

	return nil
}


func run() error {
	dir, err := os.MkdirTemp("", "fdfacts")

	if err != nil {
		return err
	}

	defer os.RemoveAll(dir)

	path := filepath.Join(dir, "file")

	if err := os.WriteFile(path, nil, 0o644); err != nil {
		return err
	}

	for _, experiment := range []func(string) error{numbers, limit, offsets, flags, names, unlinked, redirections} {
		if err := experiment(path); err != nil {
			return err
		}
	}

	return nil
}

func main() {
	if err := run(); err != nil {
		log.Fatal(err)
	}
}

Listing 2.40 The complete fdfacts, in Rust. code/rust/fdfacts/src/main.rs

//! Measures how the kernel hands out file descriptors and what a pair of
//! descriptors shares: the number an open returns, the file offset, the
//! status flags, and the close-on-exec flag. It ends with the redirection
//! `> file 2>&1`, done once with a shared open file description and once with
//! separate ones.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p fdfacts
//! ```

use std::fs::{self, File, OpenOptions};
use std::io::{self, Read};
use std::os::fd::{AsFd, AsRawFd};
use std::os::unix::fs::MetadataExt;
use std::path::Path;
use std::process::Command;

use nix::fcntl::{FcntlArg, FdFlag, OFlag, fcntl, open};
use nix::sys::resource::{Resource, getrlimit, setrlimit};
use nix::sys::stat::Mode;
use nix::unistd::{Whence, close, dup, lseek, write};

/// Closes descriptor 0, the standard input, and opens a file. The kernel
/// gives an open the lowest number that is free.
fn numbers(path: &Path) -> io::Result<()> {
    close(0)?;

    let fd = open(path, OFlag::O_RDONLY, Mode::empty())?;
    println!(
        "descriptor 0 closed: the next open returns {}",
        fd.as_raw_fd()
    );

    Ok(())
}

/// Lowers the limit on open descriptors to 32 and opens a file until the
/// kernel refuses.
fn limit(path: &Path) -> io::Result<()> {
    let (soft, hard) = getrlimit(Resource::RLIMIT_NOFILE)?;

    setrlimit(Resource::RLIMIT_NOFILE, 32, hard)?;

    let mut opened = Vec::new();

    let error = loop {
        match open(path, OFlag::O_RDONLY, Mode::empty()) {
            Ok(fd) => opened.push(fd),
            Err(error) => break error,
        }
    };

    let highest = opened.iter().map(AsRawFd::as_raw_fd).fold(-1, i32::max);

    println!(
        "limit of 32 descriptors: the highest number obtained is {highest}, the next open fails with {error:?}"
    );

    drop(opened);
    setrlimit(Resource::RLIMIT_NOFILE, soft, hard)?;

    Ok(())
}

/// Returns the file offset that a descriptor refers to.
fn offset(fd: impl AsFd) -> nix::Result<i64> {
    lseek(fd, 0, Whence::SeekCur)
}

/// Writes through one descriptor and reads the offset through a duplicate of
/// it and through a second open of the same file.
fn offsets(path: &Path) -> io::Result<()> {
    let first = open(path, OFlag::O_WRONLY, Mode::empty())?;
    let duplicate = dup(&first)?;
    let second = open(path, OFlag::O_WRONLY, Mode::empty())?;

    write(&first, b"hello")?;

    let shared = offset(&duplicate)?;
    let separate = offset(&second)?;

    println!(
        "after a write of 5 bytes: offset through the duplicate {shared}, through a second open {separate}"
    );

    Ok(())
}

/// Sets `O_APPEND` and close-on-exec through a duplicate and reads both
/// through the original descriptor.
fn flags(path: &Path) -> io::Result<()> {
    let first = open(path, OFlag::O_WRONLY, Mode::empty())?;
    let duplicate = dup(&first)?;

    fcntl(&duplicate, FcntlArg::F_SETFL(OFlag::O_APPEND))?;
    fcntl(&duplicate, FcntlArg::F_SETFD(FdFlag::FD_CLOEXEC))?;

    let status = OFlag::from_bits_truncate(fcntl(&first, FcntlArg::F_GETFL)?);
    let descriptor = FdFlag::from_bits_truncate(fcntl(&first, FcntlArg::F_GETFD)?);

    println!(
        "set on the duplicate, read on the original: O_APPEND {}, close-on-exec {}",
        status.contains(OFlag::O_APPEND),
        descriptor.contains(FdFlag::FD_CLOEXEC)
    );

    Ok(())
}

/// Gives a file a second name and compares the inodes behind the names.
fn names(path: &Path) -> io::Result<()> {
    let (first, second) = (path.with_extension("first"), path.with_extension("second"));

    fs::write(&first, "hello")?;
    fs::hard_link(&first, &second)?;

    let (a, b) = (fs::metadata(&first)?, fs::metadata(&second)?);

    println!(
        "a second name for the file: same inode {}, link count {}",
        a.ino() == b.ino(),
        b.nlink()
    );

    fs::remove_file(&first)?;
    fs::remove_file(&second)
}

/// Opens a file, removes its only name, and reads through the descriptor.
fn unlinked(path: &Path) -> io::Result<()> {
    let name = path.with_extension("unlinked");

    fs::write(&name, "hello")?;

    let mut file = File::open(&name)?;

    fs::remove_file(&name)?;

    let links = file.metadata()?.nlink();
    let mut text = String::new();

    file.read_to_string(&mut text)?;

    println!(
        "the name removed, the file still open: link count {links}, read through the descriptor returns {text:?}"
    );

    Ok(())
}

/// Runs a command that writes "out" to descriptor 1 and then "err" to
/// descriptor 2, with both descriptors on the same file, and returns the
/// file's contents.
fn redirect(path: &Path, stdout: File, stderr: File) -> io::Result<String> {
    Command::new("sh")
        .args(["-c", "echo out; echo err >&2"])
        .stdout(stdout)
        .stderr(stderr)
        .status()?;

    let text = fs::read_to_string(path)?;
    let words: Vec<&str> = text.split_whitespace().collect();

    Ok(words.join(" "))
}

/// Compares `> file 2>&1`, where descriptor 2 is a duplicate of descriptor 1,
/// with `> file 2> file`, where the file is opened once for each.
fn redirections(path: &Path) -> io::Result<()> {
    let stdout = File::create(path)?;
    let stderr = stdout.try_clone()?;

    let text = redirect(path, stdout, stderr)?;
    println!("> file 2>&1: the file holds {text:?}");

    let first = File::create(path)?;
    let second = OpenOptions::new().write(true).open(path)?;

    let text = redirect(path, first, second)?;
    println!("> file 2> file: the file holds {text:?}");

    Ok(())
}

fn main() -> io::Result<()> {
    let dir = std::env::temp_dir().join(format!("fdfacts-{}", std::process::id()));
    let path = dir.join("file");

    fs::create_dir_all(&dir)?;
    fs::write(&path, "")?;

    let result = numbers(&path)
        .and_then(|()| limit(&path))
        .and_then(|()| offsets(&path))
        .and_then(|()| flags(&path))
        .and_then(|()| names(&path))
        .and_then(|()| unlinked(&path))
        .and_then(|()| redirections(&path));

    fs::remove_dir_all(&dir)?;

    result
}

Listing 2.41 The complete fdtable, in Go. code/go/fdtable/main.go

// Command fdtable models the kernel's file tables in user space: names that
// refer to inodes, open file descriptions that refer to an inode and hold an
// offset, and a descriptor table for each process. Every object counts the
// references to it and is released when the count reaches 0. The program
// repeats the experiments of fdfacts on the model and prints the same lines.
//
// Run it from code/go with:
//
//	go run ./fdtable
package main

import (
	"errors"
	"fmt"
	"log"
	"strings"
)


// The errors carry the names of the kernel's error codes for the same cases.
var (
	errBadDescriptor = errors.New("EBADF")
	errTooManyFiles  = errors.New("EMFILE")
	errNoEntry       = errors.New("ENOENT")
)

// An inode is a file: its number, its contents, and 2 reference counts.
// links counts the names that refer to it, and opens counts the open file
// descriptions that refer to it.
type inode struct {
	number int
	data   []byte
	links  int
	opens  int
}

// A filesystem holds the inodes and a single directory, which maps names to
// inodes.
type filesystem struct {
	names  map[string]*inode
	inodes map[int]*inode
	next   int
}

func newFilesystem() *filesystem {
	return &filesystem{names: map[string]*inode{}, inodes: map[int]*inode{}, next: 1}
}

// allocate returns a new inode with no name.
func (fs *filesystem) allocate() *inode {
	file := &inode{number: fs.next}
	fs.inodes[file.number] = file
	fs.next++

	return file
}


// create makes an empty file with one name.
func (fs *filesystem) create(name string) *inode {
	file := fs.allocate()
	file.links = 1
	fs.names[name] = file

	return file
}

// link adds a second name for an existing file.
func (fs *filesystem) link(existing, added string) error {
	file, ok := fs.names[existing]

	if !ok {
		return errNoEntry
	}

	file.links++
	fs.names[added] = file

	return nil
}

// unlink removes a name. The file itself is removed only when release finds
// no other reference to it.
func (fs *filesystem) unlink(name string) error {
	file, ok := fs.names[name]

	if !ok {
		return errNoEntry
	}

	delete(fs.names, name)
	file.links--
	fs.release(file)

	return nil
}

// release frees an inode that has no name and is not open.
func (fs *filesystem) release(file *inode) {
	if file.links == 0 && file.opens == 0 {
		delete(fs.inodes, file.number)
	}
}


// A description is an open file description: the state created by one open.
// refs counts the descriptors, in all processes, that refer to it.
type description struct {
	file       *inode
	offset     int
	appendMode bool
	refs       int
}

// A descriptor is one entry of a process's table. The close-on-exec flag
// belongs to the entry.
type descriptor struct {
	open    *description
	cloexec bool
}

// A process is its descriptor table. The index of an entry is the file
// descriptor, a nil entry is a free number, and limit is the number of
// entries the process may use.
type process struct {
	fs    *filesystem
	table []*descriptor
	limit int
}

// newProcess returns a process with descriptors 0, 1 and 2 open on a
// terminal, which is a file without a name.
func newProcess(fs *filesystem) *process {
	p := &process{fs: fs, limit: 1024}
	terminal := &description{file: fs.allocate()}
	terminal.file.opens = 1

	for range 3 {
		p.install(terminal)
	}

	return p
}


// install puts a description in the lowest free entry, counts the new
// reference, and returns the number of the entry.
func (p *process) install(open *description) (int, error) {
	fd := 0

	for fd < len(p.table) && p.table[fd] != nil {
		fd++
	}

	if fd >= p.limit {
		return 0, errTooManyFiles
	}

	if fd == len(p.table) {
		p.table = append(p.table, nil)
	}

	open.refs++
	p.table[fd] = &descriptor{open: open}

	return fd, nil
}

// lookup returns the entry of a descriptor.
func (p *process) lookup(fd int) (*descriptor, error) {
	if fd < 0 || fd >= len(p.table) || p.table[fd] == nil {
		return nil, errBadDescriptor
	}

	return p.table[fd], nil
}

// open finds the inode behind a name and creates a description for it, with
// offset 0.
func (p *process) open(name string) (int, error) {
	file, ok := p.fs.names[name]

	if !ok {
		return 0, errNoEntry
	}

	fd, err := p.install(&description{file: file})

	if err == nil {
		file.opens++
	}

	return fd, err
}

// dup makes a second descriptor for an existing description.
func (p *process) dup(fd int) (int, error) {
	entry, err := p.lookup(fd)

	if err != nil {
		return 0, err
	}

	return p.install(entry.open)
}


// close frees an entry and drops its reference to the description. The last
// reference closes the description, which drops its reference to the inode.
func (p *process) close(fd int) error {
	entry, err := p.lookup(fd)

	if err != nil {
		return err
	}

	p.table[fd] = nil
	entry.open.refs--

	if entry.open.refs == 0 {
		entry.open.file.opens--
		p.fs.release(entry.open.file)
	}

	return nil
}

// dup2 makes descriptor to refer to the description of descriptor from,
// closing whatever to referred to before.
func (p *process) dup2(from, to int) error {
	entry, err := p.lookup(from)

	if err != nil {
		return err
	}

	if from == to {
		return nil
	}

	if _, err := p.lookup(to); err == nil {
		p.close(to)
	}

	for len(p.table) <= to {
		p.table = append(p.table, nil)
	}

	entry.open.refs++
	p.table[to] = &descriptor{open: entry.open}

	return nil
}


// write copies data into the file at the description's offset and advances
// the offset. In append mode the offset first moves to the end of the file.
func (p *process) write(fd int, data string) error {
	entry, err := p.lookup(fd)

	if err != nil {
		return err
	}

	open, file := entry.open, entry.open.file

	if open.appendMode {
		open.offset = len(file.data)
	}

	end := open.offset + len(data)

	if end > len(file.data) {
		file.data = append(file.data, make([]byte, end-len(file.data))...)
	}

	copy(file.data[open.offset:], data)
	open.offset = end

	return nil
}

// read returns up to n bytes from the description's offset and advances the
// offset.
func (p *process) read(fd, n int) (string, error) {
	entry, err := p.lookup(fd)

	if err != nil {
		return "", err
	}

	open := entry.open
	end := min(open.offset+n, len(open.file.data))
	data := string(open.file.data[open.offset:end])
	open.offset = end

	return data, nil
}


// fork returns a child whose table has a copy of every entry. The copies
// refer to the parent's descriptions, and each copy is counted.
func (p *process) fork() *process {
	child := &process{fs: p.fs, limit: p.limit, table: make([]*descriptor, len(p.table))}

	for fd, entry := range p.table {
		if entry != nil {
			entry.open.refs++
			child.table[fd] = &descriptor{open: entry.open, cloexec: entry.cloexec}
		}
	}

	return child
}

// exec closes every descriptor that has the close-on-exec flag.
func (p *process) exec() {
	for fd, entry := range p.table {
		if entry != nil && entry.cloexec {
			p.close(fd)
		}
	}
}


func numbers(fs *filesystem) error {
	p := newProcess(fs)

	if err := p.close(0); err != nil {
		return err
	}

	fd, err := p.open("file")
	fmt.Printf("descriptor 0 closed: the next open returns %d\n", fd)

	return err
}

func limit(fs *filesystem) error {
	p := newProcess(fs)
	p.limit = 32
	highest := -1

	for {
		fd, err := p.open("file")

		if err != nil {
			fmt.Printf("limit of 32 descriptors: the highest number obtained is %d, the next open fails with %v\n", highest, err)

			return nil
		}

		highest = max(highest, fd)
	}
}

func offsets(fs *filesystem) error {
	p := newProcess(fs)
	first, err := p.open("file")

	if err != nil {
		return err
	}

	duplicate, err := p.dup(first)

	if err != nil {
		return err
	}

	second, err := p.open("file")

	if err != nil {
		return err
	}

	if err := p.write(first, "hello"); err != nil {
		return err
	}

	fmt.Printf("after a write of 5 bytes: offset through the duplicate %d, through a second open %d\n",
		p.table[duplicate].open.offset, p.table[second].open.offset)

	return nil
}

func flags(fs *filesystem) error {
	p := newProcess(fs)
	first, err := p.open("file")

	if err != nil {
		return err
	}

	duplicate, err := p.dup(first)

	if err != nil {
		return err
	}

	p.table[duplicate].open.appendMode = true
	p.table[duplicate].cloexec = true

	fmt.Printf("set on the duplicate, read on the original: O_APPEND %v, close-on-exec %v\n",
		p.table[first].open.appendMode, p.table[first].cloexec)

	return nil
}


func names(fs *filesystem) error {
	fs.create("first")

	if err := fs.link("first", "second"); err != nil {
		return err
	}

	a, b := fs.names["first"], fs.names["second"]
	fmt.Printf("a second name for the file: same inode %v, link count %d\n", a == b, b.links)

	return nil
}

func unlinked(fs *filesystem) error {
	p := newProcess(fs)
	file := fs.create("unlinked")
	file.data = []byte("hello")

	fd, err := p.open("unlinked")

	if err != nil {
		return err
	}

	if err := fs.unlink("unlinked"); err != nil {
		return err
	}

	data, err := p.read(fd, 16)

	if err != nil {
		return err
	}

	fmt.Printf("the name removed, the file still open: link count %d, read through the descriptor returns %q\n",
		file.links, data)

	return p.close(fd)
}


// redirect forks a child, points its descriptors 1 and 2 at the given
// descriptors, and has it write "out" and then "err". It returns the file's
// contents.
func redirect(shell *process, file *inode, stdout, stderr int) (string, error) {
	child := shell.fork()

	if err := child.dup2(stdout, 1); err != nil {
		return "", err
	}

	if err := child.dup2(stderr, 2); err != nil {
		return "", err
	}

	child.exec()

	if err := child.write(1, "out\n"); err != nil {
		return "", err
	}

	if err := child.write(2, "err\n"); err != nil {
		return "", err
	}

	return strings.Join(strings.Fields(string(file.data)), " "), nil
}

func redirections(fs *filesystem) error {
	shell := newProcess(fs)
	file := fs.create("output")
	stdout, err := shell.open("output")

	if err != nil {
		return err
	}

	text, err := redirect(shell, file, stdout, stdout)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2>&1: the file holds %q\n", text)

	file.data = nil
	first, err := shell.open("output")

	if err != nil {
		return err
	}

	second, err := shell.open("output")

	if err != nil {
		return err
	}

	text, err = redirect(shell, file, first, second)

	if err != nil {
		return err
	}

	fmt.Printf("> file 2> file: the file holds %q\n", text)

	return nil
}


func main() {
	experiments := []func(*filesystem) error{numbers, limit, offsets, flags, names, unlinked, redirections}

	for _, experiment := range experiments {
		fs := newFilesystem()
		fs.create("file")

		if err := experiment(fs); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 2.42 The complete fdtable, in Rust. code/rust/fdtable/src/main.rs

//! Models the kernel's file tables in user space: names that refer to inodes,
//! open file descriptions that refer to an inode and hold an offset, and a
//! descriptor table for each process. Every object counts the references to
//! it and is released when the count reaches 0. The program repeats the
//! experiments of fdfacts on the model and prints the same lines.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p fdtable
//! ```

use std::collections::HashMap;
use std::fmt;

/// The errors carry the names of the kernel's error codes for the same cases.
#[derive(Debug)]
enum Error {
    BadDescriptor,
    TooManyFiles,
    NoEntry,
}

impl fmt::Display for Error {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str(match self {
            Error::BadDescriptor => "EBADF",
            Error::TooManyFiles => "EMFILE",
            Error::NoEntry => "ENOENT",
        })
    }
}

/// A file: its contents and 2 reference counts. `links` counts the names
/// that refer to it, and `opens` counts the open file descriptions that refer
/// to it. Its number is its key in the kernel's table of inodes.
#[derive(Default)]
struct Inode {
    data: Vec<u8>,
    links: u32,
    opens: u32,
}

/// The kernel's tables: a single directory that maps names to inode numbers,
/// the inodes, and every open file description.
#[derive(Default)]
struct Kernel {
    names: HashMap<&'static str, u64>,
    inodes: HashMap<u64, Inode>,
    open_files: Vec<Description>,
    next: u64,
}

impl Kernel {
    /// Returns the number of a new inode with no name.
    fn allocate(&mut self) -> u64 {
        self.next += 1;
        self.inodes.insert(self.next, Inode::default());

        self.next
    }

    fn inode(&mut self, number: u64) -> Result<&mut Inode, Error> {
        self.inodes.get_mut(&number).ok_or(Error::NoEntry)
    }
}

impl Kernel {
    /// Makes an empty file with one name and returns its inode number.
    fn create(&mut self, name: &'static str) -> Result<u64, Error> {
        let number = self.allocate();

        self.inode(number)?.links = 1;
        self.names.insert(name, number);

        Ok(number)
    }

    /// Adds a second name for an existing file.
    fn link(&mut self, existing: &str, added: &'static str) -> Result<(), Error> {
        let number = *self.names.get(existing).ok_or(Error::NoEntry)?;

        self.inode(number)?.links += 1;
        self.names.insert(added, number);

        Ok(())
    }

    /// Removes a name. The file itself is removed only when `release` finds
    /// no other reference to it.
    fn unlink(&mut self, name: &str) -> Result<(), Error> {
        let number = self.names.remove(name).ok_or(Error::NoEntry)?;

        self.inode(number)?.links -= 1;
        self.release(number);

        Ok(())
    }

    /// Frees an inode that has no name and is not open.
    fn release(&mut self, number: u64) {
        if let Some(file) = self.inodes.get(&number)
            && file.links == 0
            && file.opens == 0
        {
            self.inodes.remove(&number);
        }
    }
}

/// An open file description: the state created by one open. `file` is the
/// number of its inode, and `refs` counts the descriptors, in all processes,
/// that refer to it.
struct Description {
    file: u64,
    offset: usize,
    append_mode: bool,
    refs: u32,
}

/// One entry of a process's table. `open` is the index of a description in
/// the kernel's table. The close-on-exec flag belongs to the entry.
#[derive(Clone, Copy)]
struct Descriptor {
    open: usize,
    cloexec: bool,
}

/// A process is its descriptor table. The index of an entry is the file
/// descriptor, `None` is a free number, and `limit` is the number of entries
/// the process may use.
struct Process {
    table: Vec<Option<Descriptor>>,
    limit: usize,
}

impl Kernel {
    /// Adds a description for an inode and returns its index.
    fn describe(&mut self, file: u64) -> usize {
        self.open_files.push(Description {
            file,
            offset: 0,
            append_mode: false,
            refs: 0,
        });

        self.open_files.len() - 1
    }

    /// Returns a process with descriptors 0, 1 and 2 open on a terminal,
    /// which is a file without a name.
    fn process(&mut self) -> Result<Process, Error> {
        let mut process = Process {
            table: Vec::new(),
            limit: 1024,
        };

        let terminal = self.allocate();
        let open = self.describe(terminal);

        self.inode(terminal)?.opens = 1;

        for _ in 0..3 {
            self.install(&mut process, open)?;
        }

        Ok(process)
    }
}

impl Kernel {
    /// Puts a description in the lowest free entry, counts the new reference,
    /// and returns the number of the entry.
    fn install(&mut self, process: &mut Process, open: usize) -> Result<usize, Error> {
        let fd = process
            .table
            .iter()
            .take_while(|entry| entry.is_some())
            .count();

        if fd >= process.limit {
            return Err(Error::TooManyFiles);
        }

        if fd == process.table.len() {
            process.table.push(None);
        }

        self.open_files[open].refs += 1;
        process.table[fd] = Some(Descriptor {
            open,
            cloexec: false,
        });

        Ok(fd)
    }

    /// Finds the inode behind a name and creates a description for it, with
    /// offset 0.
    fn open(&mut self, process: &mut Process, name: &str) -> Result<usize, Error> {
        let file = *self.names.get(name).ok_or(Error::NoEntry)?;
        let open = self.describe(file);
        let fd = self.install(process, open)?;

        self.inode(file)?.opens += 1;

        Ok(fd)
    }

    /// Makes a second descriptor for an existing description.
    fn dup(&mut self, process: &mut Process, fd: usize) -> Result<usize, Error> {
        let entry = process.lookup(fd)?;

        self.install(process, entry.open)
    }
}

impl Process {
    /// Returns the entry of a descriptor.
    fn lookup(&self, fd: usize) -> Result<Descriptor, Error> {
        self.table
            .get(fd)
            .copied()
            .flatten()
            .ok_or(Error::BadDescriptor)
    }
}

impl Kernel {
    /// Frees an entry and drops its reference to the description. The last
    /// reference closes the description, which drops its reference to the
    /// inode.
    fn close(&mut self, process: &mut Process, fd: usize) -> Result<(), Error> {
        let entry = process.lookup(fd)?;
        let open = &mut self.open_files[entry.open];

        process.table[fd] = None;
        open.refs -= 1;

        if open.refs == 0 {
            let file = open.file;

            self.inode(file)?.opens -= 1;
            self.release(file);
        }

        Ok(())
    }

    /// Makes descriptor `to` refer to the description of descriptor `from`,
    /// closing whatever `to` referred to before.
    fn dup2(&mut self, process: &mut Process, from: usize, to: usize) -> Result<(), Error> {
        let entry = process.lookup(from)?;

        if from == to {
            return Ok(());
        }

        if process.lookup(to).is_ok() {
            self.close(process, to)?;
        }

        if process.table.len() <= to {
            process.table.resize(to + 1, None);
        }

        self.open_files[entry.open].refs += 1;
        process.table[to] = Some(Descriptor {
            open: entry.open,
            cloexec: false,
        });

        Ok(())
    }
}

impl Kernel {
    /// Copies `data` into the file at the description's offset and advances
    /// the offset. In append mode the offset first moves to the end of the
    /// file.
    fn write(&mut self, process: &Process, fd: usize, data: &str) -> Result<(), Error> {
        let open = &mut self.open_files[process.lookup(fd)?.open];
        let file = self.inodes.get_mut(&open.file).ok_or(Error::NoEntry)?;

        if open.append_mode {
            open.offset = file.data.len();
        }

        let end = open.offset + data.len();

        if end > file.data.len() {
            file.data.resize(end, 0);
        }

        file.data[open.offset..end].copy_from_slice(data.as_bytes());
        open.offset = end;

        Ok(())
    }

    /// Returns up to `n` bytes from the description's offset and advances the
    /// offset.
    fn read(&mut self, process: &Process, fd: usize, n: usize) -> Result<String, Error> {
        let open = &mut self.open_files[process.lookup(fd)?.open];
        let file = self.inodes.get(&open.file).ok_or(Error::NoEntry)?;

        let end = (open.offset + n).min(file.data.len());
        let data = String::from_utf8_lossy(&file.data[open.offset..end]).into_owned();

        open.offset = end;

        Ok(data)
    }
}

impl Kernel {
    /// Returns a child whose table has a copy of every entry. The copies
    /// refer to the parent's descriptions, and each copy is counted.
    fn fork(&mut self, parent: &Process) -> Process {
        for entry in parent.table.iter().flatten() {
            self.open_files[entry.open].refs += 1;
        }

        Process {
            table: parent.table.clone(),
            limit: parent.limit,
        }
    }

    /// Closes every descriptor that has the close-on-exec flag.
    fn exec(&mut self, process: &mut Process) -> Result<(), Error> {
        for fd in 0..process.table.len() {
            if process.table[fd].is_some_and(|entry| entry.cloexec) {
                self.close(process, fd)?;
            }
        }

        Ok(())
    }
}

fn numbers(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;

    kernel.close(&mut process, 0)?;

    let fd = kernel.open(&mut process, "file")?;
    println!("descriptor 0 closed: the next open returns {fd}");

    Ok(())
}

fn limit(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;
    let mut highest = 0;

    process.limit = 32;

    let error = loop {
        match kernel.open(&mut process, "file") {
            Ok(fd) => highest = highest.max(fd),
            Err(error) => break error,
        }
    };

    println!(
        "limit of 32 descriptors: the highest number obtained is {highest}, the next open fails with {error}"
    );

    Ok(())
}

fn offsets(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;

    let first = kernel.open(&mut process, "file")?;
    let duplicate = kernel.dup(&mut process, first)?;
    let second = kernel.open(&mut process, "file")?;

    kernel.write(&process, first, "hello")?;

    let shared = kernel.open_files[process.lookup(duplicate)?.open].offset;
    let separate = kernel.open_files[process.lookup(second)?.open].offset;

    println!(
        "after a write of 5 bytes: offset through the duplicate {shared}, through a second open {separate}"
    );

    Ok(())
}

fn flags(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;

    let first = kernel.open(&mut process, "file")?;
    let duplicate = kernel.dup(&mut process, first)?;

    kernel.open_files[process.lookup(duplicate)?.open].append_mode = true;

    if let Some(entry) = &mut process.table[duplicate] {
        entry.cloexec = true;
    }

    let original = process.lookup(first)?;

    println!(
        "set on the duplicate, read on the original: O_APPEND {}, close-on-exec {}",
        kernel.open_files[original.open].append_mode, original.cloexec
    );

    Ok(())
}

fn names(kernel: &mut Kernel) -> Result<(), Error> {
    kernel.create("first")?;
    kernel.link("first", "second")?;

    let (a, b) = (kernel.names["first"], kernel.names["second"]);

    println!(
        "a second name for the file: same inode {}, link count {}",
        a == b,
        kernel.inode(b)?.links
    );

    Ok(())
}

fn unlinked(kernel: &mut Kernel) -> Result<(), Error> {
    let mut process = kernel.process()?;
    let file = kernel.create("unlinked")?;

    kernel.inode(file)?.data = b"hello".to_vec();

    let fd = kernel.open(&mut process, "unlinked")?;

    kernel.unlink("unlinked")?;

    let links = kernel.inode(file)?.links;
    let data = kernel.read(&process, fd, 16)?;

    println!(
        "the name removed, the file still open: link count {links}, read through the descriptor returns {data:?}"
    );

    kernel.close(&mut process, fd)
}

/// Forks a child, points its descriptors 1 and 2 at the given descriptors,
/// and has it write "out" and then "err". Returns the file's contents.
fn redirect(
    kernel: &mut Kernel,
    shell: &Process,
    file: u64,
    stdout: usize,
    stderr: usize,
) -> Result<String, Error> {
    let mut child = kernel.fork(shell);

    kernel.dup2(&mut child, stdout, 1)?;
    kernel.dup2(&mut child, stderr, 2)?;
    kernel.exec(&mut child)?;

    kernel.write(&child, 1, "out\n")?;
    kernel.write(&child, 2, "err\n")?;

    let text = String::from_utf8_lossy(&kernel.inode(file)?.data).into_owned();
    let words: Vec<&str> = text.split_whitespace().collect();

    Ok(words.join(" "))
}

fn redirections(kernel: &mut Kernel) -> Result<(), Error> {
    let mut shell = kernel.process()?;
    let file = kernel.create("output")?;
    let stdout = kernel.open(&mut shell, "output")?;

    let text = redirect(kernel, &shell, file, stdout, stdout)?;
    println!("> file 2>&1: the file holds {text:?}");

    kernel.inode(file)?.data.clear();

    let first = kernel.open(&mut shell, "output")?;
    let second = kernel.open(&mut shell, "output")?;

    let text = redirect(kernel, &shell, file, first, second)?;
    println!("> file 2> file: the file holds {text:?}");

    Ok(())
}

fn main() -> Result<(), Error> {
    let experiments = [
        numbers,
        limit,
        offsets,
        flags,
        names,
        unlinked,
        redirections,
    ];

    for experiment in experiments {
        let mut kernel = Kernel::default();

        kernel.create("file")?;
        experiment(&mut kernel)?;
    }

    Ok(())
}

References

• open(2), dup(2), fcntl(2), link(2), unlink(2), stat(2) and inode(7), Linux manual pages: https://man7.org/linux/man-pages/man7/inode.7.html
• M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010, chapters 4, 5 and 18.
• include/linux/fdtable.h, include/linux/fs.h, include/linux/dcache.h and fs/file.c, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/include/linux/fs.h
• Overview of the Linux Virtual File System, Linux kernel documentation: https://www.kernel.org/doc/html/v5.15/filesystems/vfs.html

Processes

A process is a program in execution, together with everything the kernel keeps for it: its memory, its open files, its identity for permission checks, and its place in the scheduler. The kernel represents each process by a record, struct task_struct, and identifies it by a number, the process ID or pid.

In this chapter, we will follow a process through its whole life: how it is created, how it comes to run a different program, how it ends, and how the process that created it learns the result. We will read the kernel’s record of a process field by field and find the same information under /proc. Then we will write procfacts, which measures each step on a running kernel, and proctable, a model of the kernel’s process table.

A shell runs a command

When we type ls in a shell, 4 system calls do the work, 2 in the shell and 2 in a new process.

1. The shell calls fork. The kernel creates a second process that is a copy of the shell: the same program, the same memory contents, the same open files. The new process is the child, and the shell is its parent. fork returns in both processes. In the parent it returns the pid of the child, and in the child it returns 0, which is how each of them knows which one it is.
2. The child calls exec with the path of the ls program. The kernel discards the memory of the child, loads ls in its place, and starts it. The pid does not change. It is the same process running a different program.
3. ls does its work and calls exit with a status, 0 for success.
4. Meanwhile the parent has called wait, which puts it to sleep until a child ends. wait now returns the pid of the child and its status, and the shell prints the next prompt.

Creating a process and loading a program are separate calls, and that is deliberate. Between fork and exec the child is still running the shell’s code, with the shell’s descriptors. The shell uses that interval to prepare the environment of the new program: it redirects descriptors with dup2, changes the working directory, or adjusts signal handling. Then it calls exec. The new program starts in the prepared environment and contains no code for the preparation. The redirections of chapter 2 are performed in this interval, and chapter 4 builds a shell that does it.

The kernel’s record of a process

Every process has a struct task_struct. It is one of the largest structures in the kernel, about 700 lines in include/linux/sched.h. These are the fields that this chapter and the following chapters use, as they appear in Linux 5.15.

struct task_struct {
        unsigned int __state;                  /* running, sleeping, stopped, ... */
        struct sched_entity se;                /* scheduling data, chapter 6 */
        struct mm_struct *mm;                  /* the address space, chapter 7 */
        int exit_state;                        /* EXIT_ZOMBIE after exit */
        int exit_code;                         /* the status for the parent */
        pid_t pid;                             /* the number of this task */
        pid_t tgid;                            /* the number of its process */
        struct task_struct *real_parent;       /* the process that created it */
        struct list_head children;             /* its children */
        struct list_head sibling;              /* its place in the parent's list */
        const struct cred *cred;               /* user and group, capabilities */
        char comm[TASK_COMM_LEN];              /* the name of the program */
        struct files_struct *files;            /* the descriptor table, chapter 2 */
        struct nsproxy *nsproxy;               /* its namespaces, chapter 11 */
        struct signal_struct *signal;          /* signal state, chapter 5 */
        struct sighand_struct *sighand;        /* signal handlers, chapter 5 */
        struct css_set *cgroups;               /* its control groups, chapter 12 */
        /* ... */
};

The resources are separate structures reached through pointers, and this is what makes fork and threads possible with the same code. A new task can receive a copy of a resource or a pointer to the same one, resource by resource.

Reading the record from /proc

The kernel publishes the record of every process as text files in /proc/<pid>. This is a sleep started from a shell with pid 565533:

$ ps -o pid,ppid,stat,comm
    PID    PPID STAT COMMAND
 565533  565532 S    sh
 565534  565533 S    sleep
$ grep -E 'Name|State|Tgid|Pid|PPid|Threads' /proc/565534/status
Name:	sleep
State:	S (sleeping)
Tgid:	565534
Pid:	565534
PPid:	565533
Threads:	1

Name is comm, Pid and Tgid are pid and tgid, and PPid is the pid of real_parent. State is a letter derived from __state and exit_state.

Most processes on a system are in state S at any moment. A process in state R uses CPU time, and chapter 6 describes how the kernel divides the CPUs among them.

fork

fork creates a new struct task_struct and decides, for each resource of the parent, whether the child gets a copy.

A way to remember the table: the child starts as a photograph of the parent, taken at the moment of the call. Everything that is true of the parent at that moment is true of the child, with 2 differences that let them tell each other apart: the pid, and the return value of fork. From then on they are separate processes, and a change in one is invisible to the other, except through the open files they still share.

The copy of the memory is what makes fork look expensive and why it is cheap in practice. The kernel copies the page table and leaves the pages themselves shared and read-only. A page is copied only when one of the processes writes to it. A child that calls exec immediately, the usual case, discards the address space before it has written to more than a few pages.

On Linux, fork is a special case of a more general call, clone, which takes a flag for each resource: with the flag, the child shares the parent’s structure, and without it the child gets a copy. fork is clone with no sharing flags.

exec

exec keeps the task_struct and replaces what the process runs. The kernel reads the header of the program file, which is in the ELF format on Linux, to learn which parts of the file to map into memory and at which address execution starts. It builds a new address space, places the arguments and the environment variables on a new stack, and sets the processor to the entry point.

The table explains why the preparation between fork and exec works. Everything the shell arranges in that interval is in the left column.

exit and wait

A process ends by calling exit with a status, or because a signal ends it. In both cases the kernel releases its memory and closes its descriptors at once. The task_struct remains, with exit_state set to EXIT_ZOMBIE and the result in exit_code, because the parent has not yet asked for the result. A process in this condition is a zombie. It uses no memory and no CPU. It occupies a task_struct and a pid.

The parent collects the result with wait or waitpid, and the kernel then frees the task_struct. This is called reaping the child. wait delivers both kinds of ending in a single integer.

The exit status has 8 bits. A program that calls exit(300) is reported with status 44, the low 8 bits of 300. A shell folds the integer into its own $?: the exit status itself, or 128 + n for a process ended by signal n, so a process ended by SIGTERM, signal 15, appears as 143.

A parent that does not call wait leaves a zombie for each child that ends. The zombies disappear when the parent itself ends, for the reason given next.

Orphans

A zombie can be reaped only by its parent, so every process needs a living parent. When a process ends before its children, the children are orphans, and the kernel gives them a new parent at once. The new parent is process 1, the first process of the system, unless a closer ancestor has asked for the role: a process can declare itself a subreaper with prctl(PR_SET_CHILD_SUBREAPER), and it then adopts the orphans among its descendants. Process 1 and every subreaper call wait in a loop for exactly this purpose.

Processes and threads

task_struct has 2 numbers, pid and tgid, because the kernel schedules tasks, and a process can consist of several. A thread is a task that shares the address space, the descriptor table and the signal handlers of another task. It is created by clone with the sharing flags set, where fork sets none.

All threads of a process have the same tgid, the thread group ID, and each has its own pid. The number that getpid returns, and that we call the pid of a process, is the tgid. The number of each thread is visible under /proc/<pid>/task, and the Threads line of /proc/<pid>/status counts them. The sleep above has 1. A Go program always has several, because the Go runtime starts threads of its own to run goroutines.

Measuring processes

procfacts starts child processes with the standard libraries, os/exec in Go and std::process in Rust. Both perform the creation of the child and the exec in a single call, since a program with several threads cannot safely run arbitrary code between fork and exec. The Rust version uses the nix crate with the signal feature to send a signal.

$ cargo new procfacts
$ cargo add -p procfacts nix --features signal

field reads a field of /proc/<pid>/stat, a single line with the same information as status in a fixed order. Field 3 is the state letter and field 4 is the parent’s pid.

Listing 3.1 The imports of procfacts, in Go. code/go/procfacts/main.go

import (
	"errors"
	"fmt"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"syscall"
	"time"
)

Listing 3.2 The imports of procfacts, in Rust. code/rust/procfacts/src/main.rs

use std::fs;
use std::io;
use std::os::unix::process::ExitStatusExt;
use std::path::Path;
use std::process::{self, Command};
use std::thread;
use std::time::Duration;

use nix::sys::signal::{Signal, kill};
use nix::unistd::Pid;

Listing 3.3 Reading a field of a process's record, in Go. code/go/procfacts/main.go

// field returns one field of /proc/<pid>/stat, counted from 1. The second
// field, the command name, is in parentheses and may contain spaces, so the
// fields after it are counted from the closing parenthesis.
func field(pid, n int) (string, error) {
	stat, err := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))

	if err != nil {
		return "", err
	}

	end := strings.LastIndexByte(string(stat), ')')
	rest := strings.Fields(string(stat[end+1:]))

	return rest[n-3], nil
}

// output runs a shell command and returns what it printed, without the final
// newline.
func output(script string) (string, error) {
	text, err := exec.Command("sh", "-c", script).Output()

	return strings.TrimSpace(string(text)), err
}

Listing 3.4 Reading a field of a process's record, in Rust. code/rust/procfacts/src/main.rs

/// Returns one field of `/proc/<pid>/stat`, counted from 1. The second field,
/// the command name, is in parentheses and may contain spaces, so the fields
/// after it are counted from the closing parenthesis.
fn field(pid: u32, n: usize) -> io::Result<String> {
    let stat = fs::read_to_string(format!("/proc/{pid}/stat"))?;
    let rest = stat.rsplit_once(')').map_or("", |(_, rest)| rest);

    rest.split_whitespace()
        .nth(n - 3)
        .map(str::to_owned)
        .ok_or_else(|| io::ErrorKind::InvalidData.into())
}

/// Runs a shell command and returns what it printed, without the final
/// newline.
fn output(script: &str) -> io::Result<String> {
    let output = Command::new("sh").args(["-c", script]).output()?;

    Ok(String::from_utf8_lossy(&output.stdout).trim().to_owned())
}

parent runs a shell that prints $PPID, the pid of its parent, and compares it with the pid of the program.

Listing 3.5 The parent of a child, in Go. code/go/procfacts/main.go

// parent asks a child for the number of its parent process.
func parent() error {
	text, err := output("echo $PPID")

	if err != nil {
		return err
	}

	fmt.Printf("the child's parent is this process: %v\n", text == strconv.Itoa(os.Getpid()))

	return nil
}

Listing 3.6 The parent of a child, in Rust. code/rust/procfacts/src/main.rs

/// Asks a child for the number of its parent process.
fn parent() -> io::Result<()> {
    let text = output("echo $PPID")?;

    println!(
        "the child's parent is this process: {}",
        text == process::id().to_string()
    );

    Ok(())
}

status runs a script and reports how it ended. The program calls it with exit 7, with exit 300, and with a script that sends itself SIGTERM.

Listing 3.7 Exit status and signal, in Go. code/go/procfacts/main.go

// status runs a script and reports how it terminated.
func status(script string) error {
	err := exec.Command("sh", "-c", script).Run()

	var exit *exec.ExitError

	if !errors.As(err, &exit) {
		return fmt.Errorf("%s: %v", script, err)
	}

	wait, ok := exit.Sys().(syscall.WaitStatus)

	switch {
	case ok && wait.Signaled():
		fmt.Printf("%s: killed by signal %d\n", script, wait.Signal())
	default:
		fmt.Printf("%s: exited with status %d\n", script, exit.ExitCode())
	}

	return nil
}

Listing 3.8 Exit status and signal, in Rust. code/rust/procfacts/src/main.rs

/// Runs a script and reports how it terminated.
fn status(script: &str) -> io::Result<()> {
    let status = Command::new("sh").args(["-c", script]).status()?;

    match (status.signal(), status.code()) {
        (Some(signal), _) => println!("{script}: killed by signal {signal}"),
        (None, Some(code)) => println!("{script}: exited with status {code}"),
        (None, None) => println!("{script}: {status}"),
    }

    Ok(())
}

execKeepsPid runs a shell that prints its pid, replaces itself with another shell by exec, and prints the pid again.

Listing 3.9 The pid before and after exec, in Go. code/go/procfacts/main.go

// execKeepsPid prints the process number of a shell before and after the
// shell replaces its program with exec.
func execKeepsPid() error {
	text, err := output(`echo $$; exec sh -c 'echo $$'`)

	if err != nil {
		return err
	}

	pids := strings.Fields(text)

	fmt.Printf("pid before and after exec: equal %v\n", len(pids) == 2 && pids[0] == pids[1])

	return nil
}

Listing 3.10 The pid before and after exec, in Rust. code/rust/procfacts/src/main.rs

/// Prints the process number of a shell before and after the shell replaces
/// its program with exec.
fn exec_keeps_pid() -> io::Result<()> {
    let text = output("echo $$; exec sh -c 'echo $$'")?;
    let pids: Vec<&str> = text.split_whitespace().collect();

    println!(
        "pid before and after exec: equal {}",
        pids.len() == 2 && pids[0] == pids[1]
    );

    Ok(())
}

zombie starts true, which exits immediately, and polls the state field until the child has ended. Then it waits for the child and checks whether the /proc entry still exists.

Listing 3.11 A child between exit and wait, in Go. code/go/procfacts/main.go

// zombie starts a child that exits at once, and reads its state before and
// after waiting for it.
func zombie() error {
	child := exec.Command("true")

	if err := child.Start(); err != nil {
		return err
	}

	pid := child.Process.Pid
	state, err := field(pid, 3)

	for err == nil && state != "Z" {
		time.Sleep(time.Millisecond)
		state, err = field(pid, 3)
	}

	if err != nil {
		return err
	}

	if err := child.Wait(); err != nil {
		return err
	}

	_, err = os.Stat(fmt.Sprintf("/proc/%d", pid))

	fmt.Printf("exited child: state %s before wait, /proc entry present after wait: %v\n", state, err == nil)

	return nil
}

Listing 3.12 A child between exit and wait, in Rust. code/rust/procfacts/src/main.rs

/// Starts a child that exits at once, and reads its state before and after
/// waiting for it.
fn zombie() -> io::Result<()> {
    let mut child = Command::new("true").spawn()?;
    let pid = child.id();

    let mut state = field(pid, 3)?;

    while state != "Z" {
        thread::sleep(Duration::from_millis(1));
        state = field(pid, 3)?;
    }

    child.wait()?;

    let present = Path::new(&format!("/proc/{pid}")).exists();
    println!("exited child: state {state} before wait, /proc entry present after wait: {present}");

    Ok(())
}

orphan runs a shell that starts sleep 30 in the background, prints the pid of the sleep, and exits. The sleep is then an orphan. The program reads the pid of its new parent and the name of that process.

Listing 3.13 The new parent of an orphan, in Go. code/go/procfacts/main.go

// orphan starts a shell that starts a background sleep and exits. The sleep
// outlives its parent, and the kernel gives it a new one.
func orphan() error {
	text, err := output("sleep 30 >/dev/null 2>&1 & echo $!")

	if err != nil {
		return err
	}

	pid, err := strconv.Atoi(text)

	if err != nil {
		return err
	}

	defer syscall.Kill(pid, syscall.SIGKILL)

	adopter, err := field(pid, 4)

	if err != nil {
		return err
	}

	name, err := os.ReadFile("/proc/" + adopter + "/comm")

	if err != nil {
		return err
	}

	fmt.Printf("orphan adopted by: %s\n", strings.TrimSpace(string(name)))

	return nil
}

Listing 3.14 The new parent of an orphan, in Rust. code/rust/procfacts/src/main.rs

/// Starts a shell that starts a background sleep and exits. The sleep
/// outlives its parent, and the kernel gives it a new one.
fn orphan() -> io::Result<()> {
    let text = output("sleep 30 >/dev/null 2>&1 & echo $!")?;

    let pid: u32 = text
        .parse()
        .map_err(|_| io::Error::from(io::ErrorKind::InvalidData))?;

    let adopter = field(pid, 4);
    kill(Pid::from_raw(pid as i32), Signal::SIGKILL)?;

    let name = fs::read_to_string(format!("/proc/{}/comm", adopter?))?;
    println!("orphan adopted by: {}", name.trim());

    Ok(())
}

the child's parent is this process: true
exit 7: exited with status 7
exit 300: exited with status 44
kill -TERM $$: killed by signal 15
pid before and after exec: equal true
exited child: state Z before wait, /proc entry present after wait: false
orphan adopted by: systemd

The process named systemd in the last line is the service manager of the user’s session, which has registered as a subreaper. On a system without one, the line would name process 1.

Modelling the process table

proctable is a model of the part of the kernel that this chapter describes: the table of processes, the allocation of pids, the zombie state, and the adoption of orphans.

The table maps a pid to an entry. last remembers the pid allocated most recently.

Listing 3.15 The imports of proctable, in Go. code/go/proctable/main.go

import (
	"errors"
	"fmt"
	"log"
	"slices"
	"strings"
)

Listing 3.16 The imports of proctable, in Rust. code/rust/proctable/src/main.rs

use std::collections::BTreeMap;
use std::fmt;

Listing 3.17 The process table, in Go. code/go/proctable/main.go

// The errors carry the names of the kernel's error codes for the same cases.
var (
	errAgain   = errors.New("EAGAIN: no process number is free")
	errNoChild = errors.New("ECHILD: the caller has no child to wait for")
	errSearch  = errors.New("ESRCH: no such process")
)

// initPid is the process that adopts orphans.
const initPid = 1

type state byte

const (
	running state = 'R'
	zombie  state = 'Z'
)

// A process is one entry of the table.
type process struct {
	parent int
	state  state
	status int
}

// A table maps process numbers to entries. last is the number given out most
// recently, and pidMax is the largest number allowed.
type table struct {
	procs  map[int]*process
	last   int
	pidMax int
}

func newTable(pidMax int) *table {
	return &table{
		procs:  map[int]*process{initPid: {state: running}},
		last:   initPid,
		pidMax: pidMax,
	}
}

Listing 3.18 The process table, in Rust. code/rust/proctable/src/main.rs

/// The errors carry the names of the kernel's error codes for the same cases.
#[derive(Debug)]
enum Error {
    Again,
    NoChild,
    Search,
}

impl fmt::Display for Error {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str(match self {
            Error::Again => "EAGAIN: no process number is free",
            Error::NoChild => "ECHILD: the caller has no child to wait for",
            Error::Search => "ESRCH: no such process",
        })
    }
}

impl std::error::Error for Error {}

/// The process that adopts orphans.
const INIT_PID: u32 = 1;

#[derive(Clone, Copy, PartialEq)]
enum State {
    Running,
    Zombie,
}

/// One entry of the table.
struct Process {
    parent: u32,
    state: State,
    status: i32,
}

/// Maps process numbers to entries. `last` is the number given out most
/// recently, and `pid_max` is the largest number allowed.
struct Table {
    procs: BTreeMap<u32, Process>,
    last: u32,
    pid_max: u32,
}

impl Table {
    fn new(pid_max: u32) -> Table {
        let init = Process {
            parent: 0,
            state: State::Running,
            status: 0,
        };

        Table {
            procs: BTreeMap::from([(INIT_PID, init)]),
            last: INIT_PID,
            pid_max,
        }
    }
}

allocate searches upward from last and wraps around at pidMax. A pid that has been freed is therefore reused only after all higher numbers have been tried. The delay protects programs that hold a pid: a process that stored the pid of another, to send it a signal later, would otherwise reach an unrelated new process if the number were reused quickly. /proc/sys/kernel/pid_max sets the limit, 4194304 on this machine.

Listing 3.19 Allocating a pid and creating a child, in Go. code/go/proctable/main.go

// allocate returns the first free number after last, wrapping around from
// pidMax to the number after initPid.
func (t *table) allocate() (int, error) {
	pid := t.last

	for range t.pidMax {
		pid++

		if pid > t.pidMax {
			pid = initPid + 1
		}

		if _, used := t.procs[pid]; !used {
			t.last = pid

			return pid, nil
		}
	}

	return 0, errAgain
}

// fork creates a child of parent and returns the child's number.
func (t *table) fork(parent int) (int, error) {
	if _, ok := t.procs[parent]; !ok {
		return 0, errSearch
	}

	pid, err := t.allocate()

	if err != nil {
		return 0, err
	}

	t.procs[pid] = &process{parent: parent, state: running}

	return pid, nil
}

Listing 3.20 Allocating a pid and creating a child, in Rust. code/rust/proctable/src/main.rs

impl Table {
    /// Returns the first free number after `last`, wrapping around from
    /// `pid_max` to the number after `INIT_PID`.
    fn allocate(&mut self) -> Result<u32, Error> {
        let mut pid = self.last;

        for _ in 0..self.pid_max {
            pid += 1;

            if pid > self.pid_max {
                pid = INIT_PID + 1;
            }

            if !self.procs.contains_key(&pid) {
                self.last = pid;

                return Ok(pid);
            }
        }

        Err(Error::Again)
    }

    /// Creates a child of `parent` and returns the child's number.
    fn fork(&mut self, parent: u32) -> Result<u32, Error> {
        if !self.procs.contains_key(&parent) {
            return Err(Error::Search);
        }

        let pid = self.allocate()?;

        let child = Process {
            parent,
            state: State::Running,
            status: 0,
        };

        self.procs.insert(pid, child);

        Ok(pid)
    }
}

exit keeps the entry and changes its state. It then gives every child of the ended process to process 1.

Listing 3.21 Ending a process, in Go. code/go/proctable/main.go

// exit ends a process. The entry stays in the table as a zombie that holds
// the exit status, and the children of the process become children of init.
func (t *table) exit(pid, status int) error {
	p, ok := t.procs[pid]

	if !ok {
		return errSearch
	}

	p.state, p.status = zombie, status

	for _, child := range t.procs {
		if child.parent == pid {
			child.parent = initPid
		}
	}

	return nil
}

Listing 3.22 Ending a process, in Rust. code/rust/proctable/src/main.rs

impl Table {
    /// Ends a process. The entry stays in the table as a zombie that holds
    /// the exit status, and the children of the process become children of
    /// init.
    fn exit(&mut self, pid: u32, status: i32) -> Result<(), Error> {
        let process = self.procs.get_mut(&pid).ok_or(Error::Search)?;

        process.state = State::Zombie;
        process.status = status;

        for child in self.procs.values_mut() {
            if child.parent == pid {
                child.parent = INIT_PID;
            }
        }

        Ok(())
    }
}

wait has 3 outcomes. A zombie child exists, and it is removed and returned. Children exist and none has ended, where the kernel would put the caller to sleep. Or the caller has no children, which is the error ECHILD.

Listing 3.23 Collecting a child, in Go. code/go/proctable/main.go

// wait removes a zombie child of parent from the table and returns its number
// and exit status. ok is false when parent has children and none of them has
// exited, the case in which the kernel suspends the caller.
func (t *table) wait(parent int) (pid, status int, ok bool, err error) {
	children := 0

	for _, pid := range t.pids() {
		child := t.procs[pid]

		if child.parent != parent {
			continue
		}

		children++

		if child.state == zombie {
			delete(t.procs, pid)

			return pid, child.status, true, nil
		}
	}

	if children == 0 {
		return 0, 0, false, errNoChild
	}

	return 0, 0, false, nil
}

// pids returns the process numbers in use, in increasing order.
func (t *table) pids() []int {
	pids := make([]int, 0, len(t.procs))

	for pid := range t.procs {
		pids = append(pids, pid)
	}

	slices.Sort(pids)

	return pids
}

Listing 3.24 Collecting a child, in Rust. code/rust/proctable/src/main.rs

impl Table {
    /// Removes a zombie child of `parent` from the table and returns its
    /// number and exit status. The result is `None` when `parent` has
    /// children and none of them has exited, the case in which the kernel
    /// suspends the caller.
    fn wait(&mut self, parent: u32) -> Result<Option<(u32, i32)>, Error> {
        let mut children = self
            .procs
            .iter()
            .filter(|(_, child)| child.parent == parent)
            .peekable();

        if children.peek().is_none() {
            return Err(Error::NoChild);
        }

        let Some((&pid, child)) = children.find(|(_, child)| child.state == State::Zombie) else {
            return Ok(None);
        };

        let status = child.status;
        self.procs.remove(&pid);

        Ok(Some((pid, status)))
    }
}

The program applies a sequence of calls to a table with pidMax set to 6 and prints the table after each call. An entry is printed as pid(parent)state.

Listing 3.25 The trace, in Go. code/go/proctable/main.go

// String renders the table as pid(parent)state for each entry.
func (t *table) String() string {
	var cells []string

	for _, pid := range t.pids() {
		p := t.procs[pid]
		cells = append(cells, fmt.Sprintf("%d(%d)%c", pid, p.parent, p.state))
	}

	return strings.Join(cells, " ")
}

// A call is one step of the trace: an operation, the process it applies to,
// and for exit the status.
type call struct {
	op     string
	pid    int
	status int
}

// apply performs a call and describes its result.
func (t *table) apply(c call) (string, error) {
	switch c.op {
	case "fork":
		child, err := t.fork(c.pid)

		return fmt.Sprintf("fork by %d: pid %d", c.pid, child), err

	case "exit":
		return fmt.Sprintf("exit of %d, status %d", c.pid, c.status), t.exit(c.pid, c.status)

	default:
		child, status, ok, err := t.wait(c.pid)

		if !ok {
			return fmt.Sprintf("wait by %d: would block", c.pid), err
		}

		return fmt.Sprintf("wait by %d: pid %d, status %d", c.pid, child, status), nil
	}
}

func main() {
	t := newTable(6)

	calls := []call{
		{op: "fork", pid: 1},
		{op: "fork", pid: 2},
		{op: "fork", pid: 3},
		{op: "wait", pid: 2},
		{op: "exit", pid: 3},
		{op: "wait", pid: 2},
		{op: "fork", pid: 2},
		{op: "exit", pid: 4, status: 7},
		{op: "wait", pid: 1},
		{op: "fork", pid: 2},
		{op: "fork", pid: 2},
	}

	fmt.Printf("%-30s %s\n", "start", t)

	for _, c := range calls {
		result, err := t.apply(c)

		if err != nil {
			log.Fatal(err)
		}

		fmt.Printf("%-30s %s\n", result, t)
	}
}

Listing 3.26 The trace, in Rust. code/rust/proctable/src/main.rs

/// Renders the table as pid(parent)state for each entry.
impl fmt::Display for Table {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        let cells: Vec<String> = self
            .procs
            .iter()
            .map(|(pid, process)| {
                let state = match process.state {
                    State::Running => 'R',
                    State::Zombie => 'Z',
                };

                format!("{pid}({}){state}", process.parent)
            })
            .collect();

        f.write_str(&cells.join(" "))
    }
}

/// One step of the trace.
enum Call {
    Fork(u32),
    Exit(u32, i32),
    Wait(u32),
}

impl Table {
    /// Performs a call and describes its result.
    fn apply(&mut self, call: &Call) -> Result<String, Error> {
        match *call {
            Call::Fork(parent) => {
                let child = self.fork(parent)?;

                Ok(format!("fork by {parent}: pid {child}"))
            }
            Call::Exit(pid, status) => {
                self.exit(pid, status)?;

                Ok(format!("exit of {pid}, status {status}"))
            }
            Call::Wait(parent) => match self.wait(parent)? {
                Some((child, status)) => {
                    Ok(format!("wait by {parent}: pid {child}, status {status}"))
                }
                None => Ok(format!("wait by {parent}: would block")),
            },
        }
    }
}

fn main() -> Result<(), Error> {
    let mut table = Table::new(6);

    let calls = [
        Call::Fork(1),
        Call::Fork(2),
        Call::Fork(3),
        Call::Wait(2),
        Call::Exit(3, 0),
        Call::Wait(2),
        Call::Fork(2),
        Call::Exit(4, 7),
        Call::Wait(1),
        Call::Fork(2),
        Call::Fork(2),
    ];

    println!("{:<30} {table}", "start");

    for call in &calls {
        let result = table.apply(call)?;
        println!("{result:<30} {table}");
    }

    Ok(())
}

start                          1(0)R
fork by 1: pid 2               1(0)R 2(1)R
fork by 2: pid 3               1(0)R 2(1)R 3(2)R
fork by 3: pid 4               1(0)R 2(1)R 3(2)R 4(3)R
wait by 2: would block         1(0)R 2(1)R 3(2)R 4(3)R
exit of 3, status 0            1(0)R 2(1)R 3(2)Z 4(1)R
wait by 2: pid 3, status 0     1(0)R 2(1)R 4(1)R
fork by 2: pid 5               1(0)R 2(1)R 4(1)R 5(2)R
exit of 4, status 7            1(0)R 2(1)R 4(1)Z 5(2)R
wait by 1: pid 4, status 7     1(0)R 2(1)R 5(2)R
fork by 2: pid 6               1(0)R 2(1)R 5(2)R 6(2)R
fork by 2: pid 3               1(0)R 2(1)R 3(2)R 5(2)R 6(2)R

Inspecting processes on a live system

A zombie cannot be removed with kill, since it has already ended. The last command gives the pid of the parent, and the remedy is there: the parent has to call wait, or end.

The complete programs

Listing 3.27 The complete procfacts, in Go. code/go/procfacts/main.go

// Command procfacts measures the life cycle of a process: who its parent is,
// how its termination is reported, what exec keeps, what remains of a child
// that has exited and has not been waited for, and who becomes the parent of
// an orphan.
//
// Run it from code/go with:
//
//	go run ./procfacts
package main

import (
	"errors"
	"fmt"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"syscall"
	"time"
)


// field returns one field of /proc/<pid>/stat, counted from 1. The second
// field, the command name, is in parentheses and may contain spaces, so the
// fields after it are counted from the closing parenthesis.
func field(pid, n int) (string, error) {
	stat, err := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))

	if err != nil {
		return "", err
	}

	end := strings.LastIndexByte(string(stat), ')')
	rest := strings.Fields(string(stat[end+1:]))

	return rest[n-3], nil
}

// output runs a shell command and returns what it printed, without the final
// newline.
func output(script string) (string, error) {
	text, err := exec.Command("sh", "-c", script).Output()

	return strings.TrimSpace(string(text)), err
}


// parent asks a child for the number of its parent process.
func parent() error {
	text, err := output("echo $PPID")

	if err != nil {
		return err
	}

	fmt.Printf("the child's parent is this process: %v\n", text == strconv.Itoa(os.Getpid()))

	return nil
}


// status runs a script and reports how it terminated.
func status(script string) error {
	err := exec.Command("sh", "-c", script).Run()

	var exit *exec.ExitError

	if !errors.As(err, &exit) {
		return fmt.Errorf("%s: %v", script, err)
	}

	wait, ok := exit.Sys().(syscall.WaitStatus)

	switch {
	case ok && wait.Signaled():
		fmt.Printf("%s: killed by signal %d\n", script, wait.Signal())
	default:
		fmt.Printf("%s: exited with status %d\n", script, exit.ExitCode())
	}

	return nil
}


// execKeepsPid prints the process number of a shell before and after the
// shell replaces its program with exec.
func execKeepsPid() error {
	text, err := output(`echo $$; exec sh -c 'echo $$'`)

	if err != nil {
		return err
	}

	pids := strings.Fields(text)

	fmt.Printf("pid before and after exec: equal %v\n", len(pids) == 2 && pids[0] == pids[1])

	return nil
}


// zombie starts a child that exits at once, and reads its state before and
// after waiting for it.
func zombie() error {
	child := exec.Command("true")

	if err := child.Start(); err != nil {
		return err
	}

	pid := child.Process.Pid
	state, err := field(pid, 3)

	for err == nil && state != "Z" {
		time.Sleep(time.Millisecond)
		state, err = field(pid, 3)
	}

	if err != nil {
		return err
	}

	if err := child.Wait(); err != nil {
		return err
	}

	_, err = os.Stat(fmt.Sprintf("/proc/%d", pid))

	fmt.Printf("exited child: state %s before wait, /proc entry present after wait: %v\n", state, err == nil)

	return nil
}


// orphan starts a shell that starts a background sleep and exits. The sleep
// outlives its parent, and the kernel gives it a new one.
func orphan() error {
	text, err := output("sleep 30 >/dev/null 2>&1 & echo $!")

	if err != nil {
		return err
	}

	pid, err := strconv.Atoi(text)

	if err != nil {
		return err
	}

	defer syscall.Kill(pid, syscall.SIGKILL)

	adopter, err := field(pid, 4)

	if err != nil {
		return err
	}

	name, err := os.ReadFile("/proc/" + adopter + "/comm")

	if err != nil {
		return err
	}

	fmt.Printf("orphan adopted by: %s\n", strings.TrimSpace(string(name)))

	return nil
}


func main() {
	experiments := []func() error{
		parent,
		func() error { return status("exit 7") },
		func() error { return status("exit 300") },
		func() error { return status("kill -TERM $$") },
		execKeepsPid,
		zombie,
		orphan,
	}

	for _, experiment := range experiments {
		if err := experiment(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 3.28 The complete procfacts, in Rust. code/rust/procfacts/src/main.rs

//! Measures the life cycle of a process: who its parent is, how its
//! termination is reported, what exec keeps, what remains of a child that has
//! exited and has not been waited for, and who becomes the parent of an
//! orphan.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p procfacts
//! ```

use std::fs;
use std::io;
use std::os::unix::process::ExitStatusExt;
use std::path::Path;
use std::process::{self, Command};
use std::thread;
use std::time::Duration;

use nix::sys::signal::{Signal, kill};
use nix::unistd::Pid;

/// Returns one field of `/proc/<pid>/stat`, counted from 1. The second field,
/// the command name, is in parentheses and may contain spaces, so the fields
/// after it are counted from the closing parenthesis.
fn field(pid: u32, n: usize) -> io::Result<String> {
    let stat = fs::read_to_string(format!("/proc/{pid}/stat"))?;
    let rest = stat.rsplit_once(')').map_or("", |(_, rest)| rest);

    rest.split_whitespace()
        .nth(n - 3)
        .map(str::to_owned)
        .ok_or_else(|| io::ErrorKind::InvalidData.into())
}

/// Runs a shell command and returns what it printed, without the final
/// newline.
fn output(script: &str) -> io::Result<String> {
    let output = Command::new("sh").args(["-c", script]).output()?;

    Ok(String::from_utf8_lossy(&output.stdout).trim().to_owned())
}

/// Asks a child for the number of its parent process.
fn parent() -> io::Result<()> {
    let text = output("echo $PPID")?;

    println!(
        "the child's parent is this process: {}",
        text == process::id().to_string()
    );

    Ok(())
}

/// Runs a script and reports how it terminated.
fn status(script: &str) -> io::Result<()> {
    let status = Command::new("sh").args(["-c", script]).status()?;

    match (status.signal(), status.code()) {
        (Some(signal), _) => println!("{script}: killed by signal {signal}"),
        (None, Some(code)) => println!("{script}: exited with status {code}"),
        (None, None) => println!("{script}: {status}"),
    }

    Ok(())
}

/// Prints the process number of a shell before and after the shell replaces
/// its program with exec.
fn exec_keeps_pid() -> io::Result<()> {
    let text = output("echo $$; exec sh -c 'echo $$'")?;
    let pids: Vec<&str> = text.split_whitespace().collect();

    println!(
        "pid before and after exec: equal {}",
        pids.len() == 2 && pids[0] == pids[1]
    );

    Ok(())
}

/// Starts a child that exits at once, and reads its state before and after
/// waiting for it.
fn zombie() -> io::Result<()> {
    let mut child = Command::new("true").spawn()?;
    let pid = child.id();

    let mut state = field(pid, 3)?;

    while state != "Z" {
        thread::sleep(Duration::from_millis(1));
        state = field(pid, 3)?;
    }

    child.wait()?;

    let present = Path::new(&format!("/proc/{pid}")).exists();
    println!("exited child: state {state} before wait, /proc entry present after wait: {present}");

    Ok(())
}

/// Starts a shell that starts a background sleep and exits. The sleep
/// outlives its parent, and the kernel gives it a new one.
fn orphan() -> io::Result<()> {
    let text = output("sleep 30 >/dev/null 2>&1 & echo $!")?;

    let pid: u32 = text
        .parse()
        .map_err(|_| io::Error::from(io::ErrorKind::InvalidData))?;

    let adopter = field(pid, 4);
    kill(Pid::from_raw(pid as i32), Signal::SIGKILL)?;

    let name = fs::read_to_string(format!("/proc/{}/comm", adopter?))?;
    println!("orphan adopted by: {}", name.trim());

    Ok(())
}

fn main() -> io::Result<()> {
    parent()?;
    status("exit 7")?;
    status("exit 300")?;
    status("kill -TERM $$")?;
    exec_keeps_pid()?;
    zombie()?;
    orphan()
}

Listing 3.29 The complete proctable, in Go. code/go/proctable/main.go

// Command proctable models the kernel's process table: process numbers handed
// out in a cycle, a parent for every process, a zombie state between exit and
// wait, and a new parent for orphans. It applies a sequence of calls and
// prints the table after each.
//
// Run it from code/go with:
//
//	go run ./proctable
package main

import (
	"errors"
	"fmt"
	"log"
	"slices"
	"strings"
)


// The errors carry the names of the kernel's error codes for the same cases.
var (
	errAgain   = errors.New("EAGAIN: no process number is free")
	errNoChild = errors.New("ECHILD: the caller has no child to wait for")
	errSearch  = errors.New("ESRCH: no such process")
)

// initPid is the process that adopts orphans.
const initPid = 1

type state byte

const (
	running state = 'R'
	zombie  state = 'Z'
)

// A process is one entry of the table.
type process struct {
	parent int
	state  state
	status int
}

// A table maps process numbers to entries. last is the number given out most
// recently, and pidMax is the largest number allowed.
type table struct {
	procs  map[int]*process
	last   int
	pidMax int
}

func newTable(pidMax int) *table {
	return &table{
		procs:  map[int]*process{initPid: {state: running}},
		last:   initPid,
		pidMax: pidMax,
	}
}


// allocate returns the first free number after last, wrapping around from
// pidMax to the number after initPid.
func (t *table) allocate() (int, error) {
	pid := t.last

	for range t.pidMax {
		pid++

		if pid > t.pidMax {
			pid = initPid + 1
		}

		if _, used := t.procs[pid]; !used {
			t.last = pid

			return pid, nil
		}
	}

	return 0, errAgain
}

// fork creates a child of parent and returns the child's number.
func (t *table) fork(parent int) (int, error) {
	if _, ok := t.procs[parent]; !ok {
		return 0, errSearch
	}

	pid, err := t.allocate()

	if err != nil {
		return 0, err
	}

	t.procs[pid] = &process{parent: parent, state: running}

	return pid, nil
}


// exit ends a process. The entry stays in the table as a zombie that holds
// the exit status, and the children of the process become children of init.
func (t *table) exit(pid, status int) error {
	p, ok := t.procs[pid]

	if !ok {
		return errSearch
	}

	p.state, p.status = zombie, status

	for _, child := range t.procs {
		if child.parent == pid {
			child.parent = initPid
		}
	}

	return nil
}


// wait removes a zombie child of parent from the table and returns its number
// and exit status. ok is false when parent has children and none of them has
// exited, the case in which the kernel suspends the caller.
func (t *table) wait(parent int) (pid, status int, ok bool, err error) {
	children := 0

	for _, pid := range t.pids() {
		child := t.procs[pid]

		if child.parent != parent {
			continue
		}

		children++

		if child.state == zombie {
			delete(t.procs, pid)

			return pid, child.status, true, nil
		}
	}

	if children == 0 {
		return 0, 0, false, errNoChild
	}

	return 0, 0, false, nil
}

// pids returns the process numbers in use, in increasing order.
func (t *table) pids() []int {
	pids := make([]int, 0, len(t.procs))

	for pid := range t.procs {
		pids = append(pids, pid)
	}

	slices.Sort(pids)

	return pids
}


// String renders the table as pid(parent)state for each entry.
func (t *table) String() string {
	var cells []string

	for _, pid := range t.pids() {
		p := t.procs[pid]
		cells = append(cells, fmt.Sprintf("%d(%d)%c", pid, p.parent, p.state))
	}

	return strings.Join(cells, " ")
}

// A call is one step of the trace: an operation, the process it applies to,
// and for exit the status.
type call struct {
	op     string
	pid    int
	status int
}

// apply performs a call and describes its result.
func (t *table) apply(c call) (string, error) {
	switch c.op {
	case "fork":
		child, err := t.fork(c.pid)

		return fmt.Sprintf("fork by %d: pid %d", c.pid, child), err

	case "exit":
		return fmt.Sprintf("exit of %d, status %d", c.pid, c.status), t.exit(c.pid, c.status)

	default:
		child, status, ok, err := t.wait(c.pid)

		if !ok {
			return fmt.Sprintf("wait by %d: would block", c.pid), err
		}

		return fmt.Sprintf("wait by %d: pid %d, status %d", c.pid, child, status), nil
	}
}

func main() {
	t := newTable(6)

	calls := []call{
		{op: "fork", pid: 1},
		{op: "fork", pid: 2},
		{op: "fork", pid: 3},
		{op: "wait", pid: 2},
		{op: "exit", pid: 3},
		{op: "wait", pid: 2},
		{op: "fork", pid: 2},
		{op: "exit", pid: 4, status: 7},
		{op: "wait", pid: 1},
		{op: "fork", pid: 2},
		{op: "fork", pid: 2},
	}

	fmt.Printf("%-30s %s\n", "start", t)

	for _, c := range calls {
		result, err := t.apply(c)

		if err != nil {
			log.Fatal(err)
		}

		fmt.Printf("%-30s %s\n", result, t)
	}
}

Listing 3.30 The complete proctable, in Rust. code/rust/proctable/src/main.rs

//! Models the kernel's process table: process numbers handed out in a cycle,
//! a parent for every process, a zombie state between exit and wait, and a
//! new parent for orphans. It applies a sequence of calls and prints the
//! table after each.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p proctable
//! ```

use std::collections::BTreeMap;
use std::fmt;

/// The errors carry the names of the kernel's error codes for the same cases.
#[derive(Debug)]
enum Error {
    Again,
    NoChild,
    Search,
}

impl fmt::Display for Error {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str(match self {
            Error::Again => "EAGAIN: no process number is free",
            Error::NoChild => "ECHILD: the caller has no child to wait for",
            Error::Search => "ESRCH: no such process",
        })
    }
}

impl std::error::Error for Error {}

/// The process that adopts orphans.
const INIT_PID: u32 = 1;

#[derive(Clone, Copy, PartialEq)]
enum State {
    Running,
    Zombie,
}

/// One entry of the table.
struct Process {
    parent: u32,
    state: State,
    status: i32,
}

/// Maps process numbers to entries. `last` is the number given out most
/// recently, and `pid_max` is the largest number allowed.
struct Table {
    procs: BTreeMap<u32, Process>,
    last: u32,
    pid_max: u32,
}

impl Table {
    fn new(pid_max: u32) -> Table {
        let init = Process {
            parent: 0,
            state: State::Running,
            status: 0,
        };

        Table {
            procs: BTreeMap::from([(INIT_PID, init)]),
            last: INIT_PID,
            pid_max,
        }
    }
}

impl Table {
    /// Returns the first free number after `last`, wrapping around from
    /// `pid_max` to the number after `INIT_PID`.
    fn allocate(&mut self) -> Result<u32, Error> {
        let mut pid = self.last;

        for _ in 0..self.pid_max {
            pid += 1;

            if pid > self.pid_max {
                pid = INIT_PID + 1;
            }

            if !self.procs.contains_key(&pid) {
                self.last = pid;

                return Ok(pid);
            }
        }

        Err(Error::Again)
    }

    /// Creates a child of `parent` and returns the child's number.
    fn fork(&mut self, parent: u32) -> Result<u32, Error> {
        if !self.procs.contains_key(&parent) {
            return Err(Error::Search);
        }

        let pid = self.allocate()?;

        let child = Process {
            parent,
            state: State::Running,
            status: 0,
        };

        self.procs.insert(pid, child);

        Ok(pid)
    }
}

impl Table {
    /// Ends a process. The entry stays in the table as a zombie that holds
    /// the exit status, and the children of the process become children of
    /// init.
    fn exit(&mut self, pid: u32, status: i32) -> Result<(), Error> {
        let process = self.procs.get_mut(&pid).ok_or(Error::Search)?;

        process.state = State::Zombie;
        process.status = status;

        for child in self.procs.values_mut() {
            if child.parent == pid {
                child.parent = INIT_PID;
            }
        }

        Ok(())
    }
}

impl Table {
    /// Removes a zombie child of `parent` from the table and returns its
    /// number and exit status. The result is `None` when `parent` has
    /// children and none of them has exited, the case in which the kernel
    /// suspends the caller.
    fn wait(&mut self, parent: u32) -> Result<Option<(u32, i32)>, Error> {
        let mut children = self
            .procs
            .iter()
            .filter(|(_, child)| child.parent == parent)
            .peekable();

        if children.peek().is_none() {
            return Err(Error::NoChild);
        }

        let Some((&pid, child)) = children.find(|(_, child)| child.state == State::Zombie) else {
            return Ok(None);
        };

        let status = child.status;
        self.procs.remove(&pid);

        Ok(Some((pid, status)))
    }
}

/// Renders the table as pid(parent)state for each entry.
impl fmt::Display for Table {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        let cells: Vec<String> = self
            .procs
            .iter()
            .map(|(pid, process)| {
                let state = match process.state {
                    State::Running => 'R',
                    State::Zombie => 'Z',
                };

                format!("{pid}({}){state}", process.parent)
            })
            .collect();

        f.write_str(&cells.join(" "))
    }
}

/// One step of the trace.
enum Call {
    Fork(u32),
    Exit(u32, i32),
    Wait(u32),
}

impl Table {
    /// Performs a call and describes its result.
    fn apply(&mut self, call: &Call) -> Result<String, Error> {
        match *call {
            Call::Fork(parent) => {
                let child = self.fork(parent)?;

                Ok(format!("fork by {parent}: pid {child}"))
            }
            Call::Exit(pid, status) => {
                self.exit(pid, status)?;

                Ok(format!("exit of {pid}, status {status}"))
            }
            Call::Wait(parent) => match self.wait(parent)? {
                Some((child, status)) => {
                    Ok(format!("wait by {parent}: pid {child}, status {status}"))
                }
                None => Ok(format!("wait by {parent}: would block")),
            },
        }
    }
}

fn main() -> Result<(), Error> {
    let mut table = Table::new(6);

    let calls = [
        Call::Fork(1),
        Call::Fork(2),
        Call::Fork(3),
        Call::Wait(2),
        Call::Exit(3, 0),
        Call::Wait(2),
        Call::Fork(2),
        Call::Exit(4, 7),
        Call::Wait(1),
        Call::Fork(2),
        Call::Fork(2),
    ];

    println!("{:<30} {table}", "start");

    for call in &calls {
        let result = table.apply(call)?;
        println!("{result:<30} {table}");
    }

    Ok(())
}

References

• fork(2), clone(2), execve(2), wait(2), prctl(2) and proc(5), Linux manual pages: https://man7.org/linux/man-pages/man2/fork.2.html
• M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010, chapters 6 and 24 to 28.
• include/linux/sched.h, kernel/fork.c, kernel/exit.c and kernel/pid.c, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/kernel/fork.c

Building a Shell

A shell is a user program that reads command lines and converts each into the system calls of the previous chapters: pipe, fork, dup2, exec and wait. It has no special privileges and no private interface to the kernel. Everything a shell does, any program can do.

In this chapter, we will write msh, a shell that runs pipelines with input and output redirection and has the built-in commands cd and exit. We will first watch a real shell at work, by tracing the system calls it makes for a single line. Then we will build msh in stages: the parser, the redirections, the launch of a pipeline, the collection of statuses, and the built-in commands.

The loop of a shell

A shell repeats 4 steps. It reads a line. It parses the line into commands. It starts a process for each command, with the descriptors of each process arranged as the line asks. It waits for the processes to end and remembers the status of the last one. A few commands, the built-in commands, are carried out by the shell itself in place of the third step.

Each element of a command line corresponds to a small number of system calls.

A real shell, traced

strace prints the system calls of a process as it makes them. With -f it follows the children as well. This is the system shell, sh, running the line seq 1 3 | wc -l > count.txt on our machine. The first column is the pid of the caller. The calls that load shared libraries are removed, and calls that were interleaved are shown in order.

$ strace -f -e trace=pipe2,clone,dup3,execve,wait4,close,openat sh -c 'seq 1 3 | wc -l > count.txt'
570272 pipe2([3, 4], 0)                                  = 0
570272 clone(... flags=...|SIGCHLD ...)                  = 570273
570272 close(4)                                          = 0
570273 close(3)                                          = 0
570273 dup3(4, 1, 0)                                     = 1
570273 close(4)                                          = 0
570273 execve("/usr/bin/seq", ["seq", "1", "3"], ...)    = 0
570272 clone(... flags=...|SIGCHLD ...)                  = 570274
570272 close(3)                                          = 0
570272 wait4(-1, ...
570274 dup3(3, 0, 0)                                     = 0
570274 close(3)                                          = 0
570274 openat(AT_FDCWD, "count.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
570274 close(1)                                          = 0
570274 dup3(3, 1, 0)                                     = 1
570274 close(3)                                          = 0
570274 execve("/usr/bin/wc", ["wc", "-l"], ...)          = 0
570273 +++ exited with 0 +++
570272 ... wait4 resumed                                 = 570273
570274 +++ exited with 0 +++
570272 wait4(-1, ...)                                    = 570274

Every line of the trace is something we have already met.

A trace is easier to read one process at a time. Follow the lines that begin with 570273 and ignore the rest: it closes 3, makes 4 its descriptor 1, closes 4, and becomes seq. The whole preparation is 3 calls. Figure 4.3 shows the result as descriptor tables.

The redirections are carried out in the children, between clone and execve. The shell’s own descriptors 0, 1 and 2 are untouched, so the next command line starts from the same state.

Finding the program, and the environment

The trace shows execve("/usr/bin/seq", ...), although the line said seq. execve takes a path, and the shell has to find one. It looks the name up in each directory listed in the variable PATH, in order, and uses the first match. A name that contains a / is used as it is.

$ type seq cd
seq is /usr/bin/seq
cd is a shell builtin

PATH is an environment variable. The environment is a list of strings of the form NAME=value that every process has. It is the third argument of execve: the trace shows /* 92 vars */ where strace abbreviated it. A program started without special arrangement receives a copy of the environment of the process that started it, which is how settings such as PATH, HOME and LANG reach every program in a session. The environment lives in the memory of the process, so a change in a child is invisible to the parent.

msh leaves the lookup to the standard library. Go’s exec.Command and Rust’s Command::new search PATH when the name has no /, and both pass the environment of msh on to the child.

Setting up

msh uses only the standard libraries.

$ cargo new msh

Parsing

A pipeline is a list of commands, and a command is a list of arguments with optional file names for its input and output. parse reads the words of a line from left to right. A | ends the current command, a redirection operator consumes the next word as a file name, and every other word is an argument.

msh requires spaces around operators and has no quoting, which keeps the parser to a single loop.

Listing 4.1 The imports of msh, in Go. code/go/msh/main.go

import (
	"bufio"
	"errors"
	"fmt"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"syscall"
)

Listing 4.2 The imports of msh, in Rust. code/rust/msh/src/main.rs

use std::env;
use std::fs::{self, File, OpenOptions};
use std::io::{self, BufRead, PipeReader, Write};
use std::os::unix::process::ExitStatusExt;
use std::process::{self, Child, Stdio};

Listing 4.3 Parsing a line into a pipeline, in Go. code/go/msh/main.go

// A command is one stage of a pipeline: its arguments and the files that
// replace its standard input and output, when the line names any.
type command struct {
	args         []string
	input        string
	output       string
	appendOutput bool
}

var errSyntax = errors.New("syntax error")

// parse splits a line into the commands of a pipeline. Words and operators
// are separated by spaces. The operators are | < > and >>.
func parse(line string) ([]command, error) {
	pipeline := []command{{}}
	words := strings.Fields(line)

	for i := 0; i < len(words); i++ {
		current := &pipeline[len(pipeline)-1]

		switch word := words[i]; word {
		case "|":
			if len(current.args) == 0 {
				return nil, errSyntax
			}

			pipeline = append(pipeline, command{})

		case "<", ">", ">>":
			i++

			if i == len(words) {
				return nil, errSyntax
			}

			if word == "<" {
				current.input = words[i]
			} else {
				current.output, current.appendOutput = words[i], word == ">>"
			}

		default:
			current.args = append(current.args, word)
		}
	}

	if len(pipeline[len(pipeline)-1].args) == 0 {
		return nil, errSyntax
	}

	return pipeline, nil
}

Listing 4.4 Parsing a line into a pipeline, in Rust. code/rust/msh/src/main.rs

/// One stage of a pipeline: its arguments and the files that replace its
/// standard input and output, when the line names any.
#[derive(Default)]
struct Command {
    args: Vec<String>,
    input: Option<String>,
    output: Option<String>,
    append_output: bool,
}

struct SyntaxError;

/// Splits a line into the commands of a pipeline. Words and operators are
/// separated by spaces. The operators are | < > and >>.
fn parse(line: &str) -> Result<Vec<Command>, SyntaxError> {
    let mut pipeline = Vec::new();
    let mut current = Command::default();
    let mut words = line.split_whitespace();

    while let Some(word) = words.next() {
        match word {
            "|" => {
                if current.args.is_empty() {
                    return Err(SyntaxError);
                }

                pipeline.push(std::mem::take(&mut current));
            }
            "<" | ">" | ">>" => {
                let file = words.next().ok_or(SyntaxError)?.to_owned();

                if word == "<" {
                    current.input = Some(file);
                } else {
                    current.output = Some(file);
                    current.append_output = word == ">>";
                }
            }
            _ => current.args.push(word.to_owned()),
        }
    }

    if current.args.is_empty() {
        return Err(SyntaxError);
    }

    pipeline.push(current);

    Ok(pipeline)
}

Redirections

redirect opens the files a command names. > truncates the file when it is opened, which is before the command runs, as the trace showed. For this reason sort < numbers > numbers empties the file before sort reads it. >> opens with O_APPEND and keeps the contents.

Listing 4.5 Opening the files of a redirection, in Go. code/go/msh/main.go

// redirect opens the files a command names and returns the files to use as
// its standard input and output. A nil result means "keep the default".
func redirect(c command) (input, output *os.File, err error) {
	if c.input != "" {
		input, err = os.Open(c.input)

		if err != nil {
			return nil, nil, err
		}
	}

	if c.output != "" {
		flags := os.O_WRONLY | os.O_CREATE | os.O_TRUNC

		if c.appendOutput {
			flags = os.O_WRONLY | os.O_CREATE | os.O_APPEND
		}

		output, err = os.OpenFile(c.output, flags, 0o644)

		if err != nil {
			return nil, nil, err
		}
	}

	return input, output, nil
}

Listing 4.6 Opening the files of a redirection, in Rust. code/rust/msh/src/main.rs

/// Opens the files a command names and returns the files to use as its
/// standard input and output. `None` means "keep the default".
fn redirect(command: &Command) -> io::Result<(Option<File>, Option<File>)> {
    let input = command.input.as_ref().map(File::open).transpose()?;

    let output = command
        .output
        .as_ref()
        .map(|path| {
            OpenOptions::new()
                .write(true)
                .create(true)
                .append(command.append_output)
                .truncate(!command.append_output)
                .open(path)
        })
        .transpose()?;

    Ok((input, output))
}

Launching a pipeline

launch walks the pipeline from left to right. For each command except the last it creates a pipe, gives the write end to this command as descriptor 1, and carries the read end to the next iteration, where it becomes descriptor 0 of the next command. A redirection takes precedence over the pipe: when a command names a file for a descriptor, the file replaces the pipe end on that descriptor.

After a child has started, the shell closes every descriptor it opened for that child, as sh did in the trace. The rule from chapter 1 applies: a write end that stays open in the shell keeps the write count of the pipe above 0, and the next command waits for an end of the stream that is not reported.

The standard libraries perform the clone, the dup3 calls and the execve in a single call, Start in Go and spawn in Rust. We name the files for descriptors 0 and 1 of the child, and the library arranges them in the child.

Listing 4.7 Starting the commands of a pipeline, in Go. code/go/msh/main.go

// launch starts every command of a pipeline and returns the commands that
// started. Command i reads from the pipe that command i-1 writes to, unless
// a redirection replaces that descriptor.
func launch(pipeline []command) []*exec.Cmd {
	var started []*exec.Cmd
	var fromPrevious *os.File

	for i, c := range pipeline {
		stdin, stdout := os.Stdin, os.Stdout
		var toNext, nextInput *os.File

		if fromPrevious != nil {
			stdin = fromPrevious
		}

		if i < len(pipeline)-1 {
			reader, writer, err := os.Pipe()

			if err != nil {
				fmt.Fprintln(os.Stderr, "msh:", err)

				break
			}

			stdout, toNext, nextInput = writer, writer, reader
		}

		input, output, err := redirect(c)

		if input != nil {
			stdin = input
		}

		if output != nil {
			stdout = output
		}

		if err == nil {
			child := exec.Command(c.args[0], c.args[1:]...)
			child.Stdin, child.Stdout, child.Stderr = stdin, stdout, os.Stderr
			err = child.Start()

			if err == nil {
				started = append(started, child)
			}
		}

		if errors.Is(err, exec.ErrNotFound) {
			fmt.Fprintf(os.Stderr, "msh: %s: command not found\n", c.args[0])
		} else if err != nil {
			fmt.Fprintln(os.Stderr, "msh:", err)
		}

		// The child has its own copies. The shell closes every descriptor it
		// opened for this command.
		for _, file := range []*os.File{input, output, toNext, fromPrevious} {
			if file != nil {
				file.Close()
			}
		}

		fromPrevious = nextInput
	}

	return started
}

Listing 4.8 Starting the commands of a pipeline, in Rust. code/rust/msh/src/main.rs

/// Starts every command of a pipeline and returns the children that started.
/// Command `i` reads from the pipe that command `i - 1` writes to, unless a
/// redirection replaces that descriptor.
fn launch(pipeline: &[Command]) -> Vec<Child> {
    let mut started = Vec::new();
    let mut from_previous: Option<PipeReader> = None;

    for (i, command) in pipeline.iter().enumerate() {
        let mut stdin = from_previous
            .take()
            .map_or_else(Stdio::inherit, Stdio::from);
        let mut stdout = Stdio::inherit();

        if i + 1 < pipeline.len() {
            let Ok((reader, writer)) = io::pipe() else {
                eprintln!("msh: cannot create a pipe");

                break;
            };

            stdout = writer.into();
            from_previous = Some(reader);
        }

        let (input, output) = match redirect(command) {
            Ok(files) => files,
            Err(error) => {
                eprintln!("msh: {error}");

                continue;
            }
        };

        if let Some(file) = input {
            stdin = file.into();
        }

        if let Some(file) = output {
            stdout = file.into();
        }

        // The `Command` owns the shell's copies of the descriptors. It is
        // dropped when the statement ends, which closes them.
        let child = process::Command::new(&command.args[0])
            .args(&command.args[1..])
            .stdin(stdin)
            .stdout(stdout)
            .spawn();

        match child {
            Ok(child) => started.push(child),
            Err(error) if error.kind() == io::ErrorKind::NotFound => {
                eprintln!("msh: {}: command not found", command.args[0]);
            }
            Err(error) => eprintln!("msh: {error}"),
        }
    }

    started
}

Collecting statuses

The shell waits for every child. A child that is not waited for remains a zombie, as chapter 3 showed. The status of a pipeline is the status of its last command. status applies the conventions of a shell.

By convention 0 means success and any other value means failure, and the commands if and while of a full shell test exactly this number.

Listing 4.9 Waiting for the pipeline, in Go. code/go/msh/main.go

// status converts a wait status to the number a shell reports: the exit
// status, or 128 plus the signal number.
func status(child *exec.Cmd) int {
	child.Wait()

	if wait, ok := child.ProcessState.Sys().(syscall.WaitStatus); ok && wait.Signaled() {
		return 128 + int(wait.Signal())
	}

	return child.ProcessState.ExitCode()
}

// run starts a pipeline, waits for every command, and returns the status of
// the last command, or 127 when the last command did not start.
func run(pipeline []command) int {
	started := launch(pipeline)
	last := 127

	for _, child := range started {
		last = status(child)
	}

	if len(started) < len(pipeline) {
		return 127
	}

	return last
}

Listing 4.10 Waiting for the pipeline, in Rust. code/rust/msh/src/main.rs

/// Converts a wait status to the number a shell reports: the exit status, or
/// 128 plus the signal number.
fn status(child: &mut Child) -> i32 {
    let Ok(status) = child.wait() else {
        return 127;
    };

    match (status.signal(), status.code()) {
        (Some(signal), _) => 128 + signal,
        (None, Some(code)) => code,
        (None, None) => 127,
    }
}

/// Starts a pipeline, waits for every command, and returns the status of the
/// last command, or 127 when the last command did not start.
fn run(pipeline: &[Command]) -> i32 {
    let mut started = launch(pipeline);
    let mut last = 127;

    for child in &mut started {
        last = status(child);
    }

    if started.len() < pipeline.len() {
        return 127;
    }

    last
}

Built-in commands

The working directory is an attribute of a process, kept by the kernel, and a child receives a copy of it. If cd were a separate program, it would change the directory of its own process and exit, and the directory of the shell would be unchanged. The same reasoning applies to exit. Both therefore run inside the shell process.

Listing 4.11 Running a line: built-in commands and pipelines, in Go. code/go/msh/main.go

// A shell holds the state that outlives a command line.
type shell struct {
	last    int
	exiting bool
}

// execute runs one command line. cd and exit change the shell process
// itself, so they run inside it. Everything else runs in child processes.
func (s *shell) execute(line string) {
	pipeline, err := parse(line)

	if err != nil {
		fmt.Fprintln(os.Stderr, "msh:", err)
		s.last = 2

		return
	}

	switch first := pipeline[0]; {
	case len(pipeline) == 1 && first.args[0] == "cd" && len(first.args) == 2:
		s.last = 0

		if err := os.Chdir(first.args[1]); err != nil {
			fmt.Fprintln(os.Stderr, "msh:", err)
			s.last = 1
		}

	case len(pipeline) == 1 && first.args[0] == "exit":
		s.exiting = true

		if len(first.args) == 2 {
			s.last, _ = strconv.Atoi(first.args[1])
		}

	default:
		s.last = run(pipeline)
	}
}

Listing 4.12 Running a line: built-in commands and pipelines, in Rust. code/rust/msh/src/main.rs

/// The state that outlives a command line.
#[derive(Default)]
struct Shell {
    last: i32,
    exiting: bool,
}

impl Shell {
    /// Runs one command line. cd and exit change the shell process itself, so
    /// they run inside it. Everything else runs in child processes.
    fn execute(&mut self, line: &str) {
        let Ok(pipeline) = parse(line) else {
            eprintln!("msh: syntax error");
            self.last = 2;

            return;
        };

        let args: Vec<&str> = pipeline[0].args.iter().map(String::as_str).collect();

        match (pipeline.len(), args.as_slice()) {
            (1, ["cd", directory]) => {
                self.last = 0;

                if let Err(error) = env::set_current_dir(directory) {
                    eprintln!("msh: {error}");
                    self.last = 1;
                }
            }
            (1, ["exit", rest @ ..]) => {
                self.exiting = true;

                if let [code] = rest
                    && let Ok(code) = code.parse()
                {
                    self.last = code;
                }
            }
            _ => self.last = run(&pipeline),
        }
    }
}

Running a script

Without arguments, msh runs a fixed script in a temporary directory and prints the status after each line. With -i, it reads lines from standard input.

Listing 4.13 The script, and the interactive loop, in Go. code/go/msh/main.go

// script is the demonstration that msh runs without arguments.
var script = []string{
	"seq 1 5 > numbers",
	"wc -l < numbers",
	"sort -rn < numbers | head -n 2 >> numbers",
	"paste -sd, numbers",
	"false",
	"nosuchcommand",
	"cd /",
	"pwd",
	"exit 3",
	"echo not reached",
}

// demonstrate runs the script in a new temporary directory and prints the
// status after each line.
func demonstrate() error {
	dir, err := os.MkdirTemp("", "msh")

	if err != nil {
		return err
	}

	defer os.RemoveAll(dir)

	if err := os.Chdir(dir); err != nil {
		return err
	}

	s := &shell{}

	for _, line := range script {
		fmt.Println("$", line)
		s.execute(line)

		if s.exiting {
			fmt.Printf("[the shell exits with status %d]\n", s.last)

			return nil
		}

		fmt.Printf("[status %d]\n", s.last)
	}

	return nil
}

// interact reads command lines from standard input until exit or the end of
// the input, and ends the process with the last status.
func interact() {
	s := &shell{}
	lines := bufio.NewScanner(os.Stdin)

	for !s.exiting {
		fmt.Print("msh$ ")

		if !lines.Scan() {
			break
		}

		if strings.TrimSpace(lines.Text()) != "" {
			s.execute(lines.Text())
		}
	}

	os.Exit(s.last)
}

func main() {
	if len(os.Args) > 1 && os.Args[1] == "-i" {
		interact()
	}

	if err := demonstrate(); err != nil {
		log.Fatal(err)
	}
}

Listing 4.14 The script, and the interactive loop, in Rust. code/rust/msh/src/main.rs

/// The demonstration that msh runs without arguments.
const SCRIPT: [&str; 10] = [
    "seq 1 5 > numbers",
    "wc -l < numbers",
    "sort -rn < numbers | head -n 2 >> numbers",
    "paste -sd, numbers",
    "false",
    "nosuchcommand",
    "cd /",
    "pwd",
    "exit 3",
    "echo not reached",
];

/// Runs the script in a new temporary directory and prints the status after
/// each line.
fn demonstrate() -> io::Result<()> {
    let dir = env::temp_dir().join(format!("msh-{}", process::id()));

    fs::create_dir_all(&dir)?;
    env::set_current_dir(&dir)?;

    let mut shell = Shell::default();

    for line in SCRIPT {
        println!("$ {line}");
        shell.execute(line);

        if shell.exiting {
            println!("[the shell exits with status {}]", shell.last);

            break;
        }

        println!("[status {}]", shell.last);
    }

    fs::remove_dir_all(&dir)
}

/// Reads command lines from standard input until exit or the end of the
/// input, and ends the process with the last status.
fn interact() -> io::Result<()> {
    let mut shell = Shell::default();
    let mut lines = io::stdin().lock().lines();

    while !shell.exiting {
        print!("msh$ ");
        io::stdout().flush()?;

        let Some(line) = lines.next().transpose()? else {
            break;
        };

        if !line.trim().is_empty() {
            shell.execute(&line);
        }
    }

    process::exit(shell.last);
}

fn main() -> io::Result<()> {
    if env::args().nth(1).as_deref() == Some("-i") {
        interact()?;
    }

    demonstrate()
}

$ seq 1 5 > numbers
[status 0]
$ wc -l < numbers
5
[status 0]
$ sort -rn < numbers | head -n 2 >> numbers
[status 0]
$ paste -sd, numbers
1,2,3,4,5,5,4
[status 0]
$ false
[status 1]
$ nosuchcommand
msh: nosuchcommand: command not found
[status 127]
$ cd /
[status 0]
$ pwd
/
[status 0]
$ exit 3
[the shell exits with status 3]

What a full shell adds

msh passes each word of a line to the program unchanged. A shell such as bash or dash processes the words first, and it is useful to know that these steps happen in the shell, before any program starts.

The last row leads to job control. A shell places each pipeline in a process group, a set of processes that can be signalled together, and tells the terminal which group is in the foreground. When we press Ctrl-C, the terminal driver in the kernel sends SIGINT to every process of the foreground group, and the shell itself, which is in a different group, is unaffected. Signals are the subject of chapter 5.

Inspecting a shell at work

The complete program

Listing 4.15 The complete msh, in Go. code/go/msh/main.go

// Command msh is a small shell. It parses a command line into a pipeline of
// commands with redirections, starts each command with its descriptors
// arranged, waits for all of them, and reports the status of the last. The
// commands cd and exit are built in.
//
// Without arguments it runs a fixed script in a temporary directory. With -i
// it reads command lines from standard input.
//
// Run it from code/go with:
//
//	go run ./msh
package main

import (
	"bufio"
	"errors"
	"fmt"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"syscall"
)


// A command is one stage of a pipeline: its arguments and the files that
// replace its standard input and output, when the line names any.
type command struct {
	args         []string
	input        string
	output       string
	appendOutput bool
}

var errSyntax = errors.New("syntax error")

// parse splits a line into the commands of a pipeline. Words and operators
// are separated by spaces. The operators are | < > and >>.
func parse(line string) ([]command, error) {
	pipeline := []command{{}}
	words := strings.Fields(line)

	for i := 0; i < len(words); i++ {
		current := &pipeline[len(pipeline)-1]

		switch word := words[i]; word {
		case "|":
			if len(current.args) == 0 {
				return nil, errSyntax
			}

			pipeline = append(pipeline, command{})

		case "<", ">", ">>":
			i++

			if i == len(words) {
				return nil, errSyntax
			}

			if word == "<" {
				current.input = words[i]
			} else {
				current.output, current.appendOutput = words[i], word == ">>"
			}

		default:
			current.args = append(current.args, word)
		}
	}

	if len(pipeline[len(pipeline)-1].args) == 0 {
		return nil, errSyntax
	}

	return pipeline, nil
}


// redirect opens the files a command names and returns the files to use as
// its standard input and output. A nil result means "keep the default".
func redirect(c command) (input, output *os.File, err error) {
	if c.input != "" {
		input, err = os.Open(c.input)

		if err != nil {
			return nil, nil, err
		}
	}

	if c.output != "" {
		flags := os.O_WRONLY | os.O_CREATE | os.O_TRUNC

		if c.appendOutput {
			flags = os.O_WRONLY | os.O_CREATE | os.O_APPEND
		}

		output, err = os.OpenFile(c.output, flags, 0o644)

		if err != nil {
			return nil, nil, err
		}
	}

	return input, output, nil
}


// launch starts every command of a pipeline and returns the commands that
// started. Command i reads from the pipe that command i-1 writes to, unless
// a redirection replaces that descriptor.
func launch(pipeline []command) []*exec.Cmd {
	var started []*exec.Cmd
	var fromPrevious *os.File

	for i, c := range pipeline {
		stdin, stdout := os.Stdin, os.Stdout
		var toNext, nextInput *os.File

		if fromPrevious != nil {
			stdin = fromPrevious
		}

		if i < len(pipeline)-1 {
			reader, writer, err := os.Pipe()

			if err != nil {
				fmt.Fprintln(os.Stderr, "msh:", err)

				break
			}

			stdout, toNext, nextInput = writer, writer, reader
		}

		input, output, err := redirect(c)

		if input != nil {
			stdin = input
		}

		if output != nil {
			stdout = output
		}

		if err == nil {
			child := exec.Command(c.args[0], c.args[1:]...)
			child.Stdin, child.Stdout, child.Stderr = stdin, stdout, os.Stderr
			err = child.Start()

			if err == nil {
				started = append(started, child)
			}
		}

		if errors.Is(err, exec.ErrNotFound) {
			fmt.Fprintf(os.Stderr, "msh: %s: command not found\n", c.args[0])
		} else if err != nil {
			fmt.Fprintln(os.Stderr, "msh:", err)
		}

		// The child has its own copies. The shell closes every descriptor it
		// opened for this command.
		for _, file := range []*os.File{input, output, toNext, fromPrevious} {
			if file != nil {
				file.Close()
			}
		}

		fromPrevious = nextInput
	}

	return started
}


// status converts a wait status to the number a shell reports: the exit
// status, or 128 plus the signal number.
func status(child *exec.Cmd) int {
	child.Wait()

	if wait, ok := child.ProcessState.Sys().(syscall.WaitStatus); ok && wait.Signaled() {
		return 128 + int(wait.Signal())
	}

	return child.ProcessState.ExitCode()
}

// run starts a pipeline, waits for every command, and returns the status of
// the last command, or 127 when the last command did not start.
func run(pipeline []command) int {
	started := launch(pipeline)
	last := 127

	for _, child := range started {
		last = status(child)
	}

	if len(started) < len(pipeline) {
		return 127
	}

	return last
}


// A shell holds the state that outlives a command line.
type shell struct {
	last    int
	exiting bool
}

// execute runs one command line. cd and exit change the shell process
// itself, so they run inside it. Everything else runs in child processes.
func (s *shell) execute(line string) {
	pipeline, err := parse(line)

	if err != nil {
		fmt.Fprintln(os.Stderr, "msh:", err)
		s.last = 2

		return
	}

	switch first := pipeline[0]; {
	case len(pipeline) == 1 && first.args[0] == "cd" && len(first.args) == 2:
		s.last = 0

		if err := os.Chdir(first.args[1]); err != nil {
			fmt.Fprintln(os.Stderr, "msh:", err)
			s.last = 1
		}

	case len(pipeline) == 1 && first.args[0] == "exit":
		s.exiting = true

		if len(first.args) == 2 {
			s.last, _ = strconv.Atoi(first.args[1])
		}

	default:
		s.last = run(pipeline)
	}
}


// script is the demonstration that msh runs without arguments.
var script = []string{
	"seq 1 5 > numbers",
	"wc -l < numbers",
	"sort -rn < numbers | head -n 2 >> numbers",
	"paste -sd, numbers",
	"false",
	"nosuchcommand",
	"cd /",
	"pwd",
	"exit 3",
	"echo not reached",
}

// demonstrate runs the script in a new temporary directory and prints the
// status after each line.
func demonstrate() error {
	dir, err := os.MkdirTemp("", "msh")

	if err != nil {
		return err
	}

	defer os.RemoveAll(dir)

	if err := os.Chdir(dir); err != nil {
		return err
	}

	s := &shell{}

	for _, line := range script {
		fmt.Println("$", line)
		s.execute(line)

		if s.exiting {
			fmt.Printf("[the shell exits with status %d]\n", s.last)

			return nil
		}

		fmt.Printf("[status %d]\n", s.last)
	}

	return nil
}

// interact reads command lines from standard input until exit or the end of
// the input, and ends the process with the last status.
func interact() {
	s := &shell{}
	lines := bufio.NewScanner(os.Stdin)

	for !s.exiting {
		fmt.Print("msh$ ")

		if !lines.Scan() {
			break
		}

		if strings.TrimSpace(lines.Text()) != "" {
			s.execute(lines.Text())
		}
	}

	os.Exit(s.last)
}

func main() {
	if len(os.Args) > 1 && os.Args[1] == "-i" {
		interact()
	}

	if err := demonstrate(); err != nil {
		log.Fatal(err)
	}
}

Listing 4.16 The complete msh, in Rust. code/rust/msh/src/main.rs

//! A small shell. It parses a command line into a pipeline of commands with
//! redirections, starts each command with its descriptors arranged, waits for
//! all of them, and reports the status of the last. The commands cd and exit
//! are built in.
//!
//! Without arguments it runs a fixed script in a temporary directory. With -i
//! it reads command lines from standard input.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p msh
//! ```

use std::env;
use std::fs::{self, File, OpenOptions};
use std::io::{self, BufRead, PipeReader, Write};
use std::os::unix::process::ExitStatusExt;
use std::process::{self, Child, Stdio};

/// One stage of a pipeline: its arguments and the files that replace its
/// standard input and output, when the line names any.
#[derive(Default)]
struct Command {
    args: Vec<String>,
    input: Option<String>,
    output: Option<String>,
    append_output: bool,
}

struct SyntaxError;

/// Splits a line into the commands of a pipeline. Words and operators are
/// separated by spaces. The operators are | < > and >>.
fn parse(line: &str) -> Result<Vec<Command>, SyntaxError> {
    let mut pipeline = Vec::new();
    let mut current = Command::default();
    let mut words = line.split_whitespace();

    while let Some(word) = words.next() {
        match word {
            "|" => {
                if current.args.is_empty() {
                    return Err(SyntaxError);
                }

                pipeline.push(std::mem::take(&mut current));
            }
            "<" | ">" | ">>" => {
                let file = words.next().ok_or(SyntaxError)?.to_owned();

                if word == "<" {
                    current.input = Some(file);
                } else {
                    current.output = Some(file);
                    current.append_output = word == ">>";
                }
            }
            _ => current.args.push(word.to_owned()),
        }
    }

    if current.args.is_empty() {
        return Err(SyntaxError);
    }

    pipeline.push(current);

    Ok(pipeline)
}

/// Opens the files a command names and returns the files to use as its
/// standard input and output. `None` means "keep the default".
fn redirect(command: &Command) -> io::Result<(Option<File>, Option<File>)> {
    let input = command.input.as_ref().map(File::open).transpose()?;

    let output = command
        .output
        .as_ref()
        .map(|path| {
            OpenOptions::new()
                .write(true)
                .create(true)
                .append(command.append_output)
                .truncate(!command.append_output)
                .open(path)
        })
        .transpose()?;

    Ok((input, output))
}

/// Starts every command of a pipeline and returns the children that started.
/// Command `i` reads from the pipe that command `i - 1` writes to, unless a
/// redirection replaces that descriptor.
fn launch(pipeline: &[Command]) -> Vec<Child> {
    let mut started = Vec::new();
    let mut from_previous: Option<PipeReader> = None;

    for (i, command) in pipeline.iter().enumerate() {
        let mut stdin = from_previous
            .take()
            .map_or_else(Stdio::inherit, Stdio::from);
        let mut stdout = Stdio::inherit();

        if i + 1 < pipeline.len() {
            let Ok((reader, writer)) = io::pipe() else {
                eprintln!("msh: cannot create a pipe");

                break;
            };

            stdout = writer.into();
            from_previous = Some(reader);
        }

        let (input, output) = match redirect(command) {
            Ok(files) => files,
            Err(error) => {
                eprintln!("msh: {error}");

                continue;
            }
        };

        if let Some(file) = input {
            stdin = file.into();
        }

        if let Some(file) = output {
            stdout = file.into();
        }

        // The `Command` owns the shell's copies of the descriptors. It is
        // dropped when the statement ends, which closes them.
        let child = process::Command::new(&command.args[0])
            .args(&command.args[1..])
            .stdin(stdin)
            .stdout(stdout)
            .spawn();

        match child {
            Ok(child) => started.push(child),
            Err(error) if error.kind() == io::ErrorKind::NotFound => {
                eprintln!("msh: {}: command not found", command.args[0]);
            }
            Err(error) => eprintln!("msh: {error}"),
        }
    }

    started
}

/// Converts a wait status to the number a shell reports: the exit status, or
/// 128 plus the signal number.
fn status(child: &mut Child) -> i32 {
    let Ok(status) = child.wait() else {
        return 127;
    };

    match (status.signal(), status.code()) {
        (Some(signal), _) => 128 + signal,
        (None, Some(code)) => code,
        (None, None) => 127,
    }
}

/// Starts a pipeline, waits for every command, and returns the status of the
/// last command, or 127 when the last command did not start.
fn run(pipeline: &[Command]) -> i32 {
    let mut started = launch(pipeline);
    let mut last = 127;

    for child in &mut started {
        last = status(child);
    }

    if started.len() < pipeline.len() {
        return 127;
    }

    last
}

/// The state that outlives a command line.
#[derive(Default)]
struct Shell {
    last: i32,
    exiting: bool,
}

impl Shell {
    /// Runs one command line. cd and exit change the shell process itself, so
    /// they run inside it. Everything else runs in child processes.
    fn execute(&mut self, line: &str) {
        let Ok(pipeline) = parse(line) else {
            eprintln!("msh: syntax error");
            self.last = 2;

            return;
        };

        let args: Vec<&str> = pipeline[0].args.iter().map(String::as_str).collect();

        match (pipeline.len(), args.as_slice()) {
            (1, ["cd", directory]) => {
                self.last = 0;

                if let Err(error) = env::set_current_dir(directory) {
                    eprintln!("msh: {error}");
                    self.last = 1;
                }
            }
            (1, ["exit", rest @ ..]) => {
                self.exiting = true;

                if let [code] = rest
                    && let Ok(code) = code.parse()
                {
                    self.last = code;
                }
            }
            _ => self.last = run(&pipeline),
        }
    }
}

/// The demonstration that msh runs without arguments.
const SCRIPT: [&str; 10] = [
    "seq 1 5 > numbers",
    "wc -l < numbers",
    "sort -rn < numbers | head -n 2 >> numbers",
    "paste -sd, numbers",
    "false",
    "nosuchcommand",
    "cd /",
    "pwd",
    "exit 3",
    "echo not reached",
];

/// Runs the script in a new temporary directory and prints the status after
/// each line.
fn demonstrate() -> io::Result<()> {
    let dir = env::temp_dir().join(format!("msh-{}", process::id()));

    fs::create_dir_all(&dir)?;
    env::set_current_dir(&dir)?;

    let mut shell = Shell::default();

    for line in SCRIPT {
        println!("$ {line}");
        shell.execute(line);

        if shell.exiting {
            println!("[the shell exits with status {}]", shell.last);

            break;
        }

        println!("[status {}]", shell.last);
    }

    fs::remove_dir_all(&dir)
}

/// Reads command lines from standard input until exit or the end of the
/// input, and ends the process with the last status.
fn interact() -> io::Result<()> {
    let mut shell = Shell::default();
    let mut lines = io::stdin().lock().lines();

    while !shell.exiting {
        print!("msh$ ");
        io::stdout().flush()?;

        let Some(line) = lines.next().transpose()? else {
            break;
        };

        if !line.trim().is_empty() {
            shell.execute(&line);
        }
    }

    process::exit(shell.last);
}

fn main() -> io::Result<()> {
    if env::args().nth(1).as_deref() == Some("-i") {
        interact()?;
    }

    demonstrate()
}

References

• The Open Group Base Specifications, Shell Command Language: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
• execve(2), environ(7), chdir(2) and strace(1), Linux manual pages: https://man7.org/linux/man-pages/man2/execve.2.html
• M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010, chapters 6, 27, 34 and 44.

Signals

A signal is a notification that the kernel delivers to a process at a moment the process did not choose. Each signal is identified by a small number and a name. The kernel sends signals to report events: a key pressed on the terminal, a write to a pipe with no reader, the end of a child, an invalid memory access. Processes send them to each other with the kill system call.

In this chapter, we will learn what state the kernel keeps for signals and at which moment a signal takes effect. We will follow Ctrl-C from the keyboard to the end of a program, read the kernel’s structures, and see how a handler function is run. Then we will measure signal handling from outside a process with sigfacts, and write sigmodel, a model of the kernel’s signal state.

The common signals

Linux has 64 signals. Numbers 1 to 31 are the standard signals, each with a fixed meaning, and numbers 32 to 64 are the real-time signals, which programs may use for their own purposes. kill -l lists them. These are the ones a programmer meets.

For each signal a process has a disposition, also called its action: what happens when the signal is delivered. There are 3 possibilities. The default action is the one in the table. Ignore discards the signal. A handler is a function of the program that the kernel calls when the signal arrives. SIGTERM is the polite request to end, which a program can handle to save its work, and SIGKILL is the request that cannot be refused.

From Ctrl-C to the end of a program

Suppose a program is running in a terminal, blocked in a read, and we press Ctrl-C.

1. The terminal passes the byte 0x03 to the kernel’s terminal driver.
2. The driver recognises it as the interrupt character and does not put it in the input of the program. It sends SIGINT to every process of the terminal’s foreground process group, the set of processes the shell started for the current command line.
3. Sending means setting a bit: the kernel sets the bit for SIGINT in the pending mask of each of those processes. Nothing else has happened to them yet.
4. A process that is sleeping in a system call is woken so that it can notice.
5. When the process is next about to return from the kernel to user mode, the kernel checks its pending mask, finds SIGINT, clears the bit, and performs the action. The default action ends the process.

Steps 3 and 5 are separate events. Sending a signal runs in the sender, or in the kernel on behalf of a device, and only records that the signal is wanted. Delivery runs later, in the target, at the boundary between kernel mode and user mode. Every property of signals in this chapter follows from that split.

The state behind a signal

The kernel keeps 3 pieces of signal state.

Blocking a signal is different from ignoring it. An ignored signal is discarded. A blocked signal stays pending and is delivered when the program unblocks it. A program blocks a signal around a piece of code that must not be interrupted by its handler.

These are the kernel’s definitions in Linux 5.15, from include/linux/signal_types.h, include/linux/sched/signal.h and include/linux/sched.h, reduced to the fields we use.

struct sigpending {
        struct list_head list;         /* queued details of pending signals */
        sigset_t signal;               /* the pending mask: 64 bits */
};

struct sigaction {
        __sighandler_t sa_handler;     /* SIG_DFL, SIG_IGN, or a function */
        unsigned long sa_flags;        /* SA_RESTART, ... */
        sigset_t sa_mask;              /* signals blocked while the handler runs */
};

struct sighand_struct {
        struct k_sigaction action[_NSIG];      /* one action per signal, _NSIG is 64 */
        /* ... */
};

struct signal_struct {
        struct sigpending shared_pending;      /* signals sent to the process */
        /* ... */
};

struct task_struct {
        struct signal_struct *signal;
        struct sighand_struct *sighand;
        sigset_t blocked;                      /* the blocked mask */
        struct sigpending pending;             /* signals sent to this thread */
        /* ... */
};

The kernel shows these masks in /proc/<pid>/status. This is a shell after trap '' INT and trap 'echo x' USR1:

$ grep -E 'SigPnd|ShdPnd|SigBlk|SigIgn|SigCgt' /proc/$$/status
SigPnd:	0000000000000000
ShdPnd:	0000000000000000
SigBlk:	0000000000000000
SigIgn:	0000000000000002
SigCgt:	0000000000010200

SigPnd is the thread’s pending mask and ShdPnd is shared_pending. SigBlk is blocked. SigIgn and SigCgt are derived from the action table: the signals that are ignored and the signals that have a handler. The values are hexadecimal. 0x2 is bit 1, signal 2, SIGINT. 0x10200 is bit 9, signal 10, SIGUSR1, and bit 16, signal 17, SIGCHLD, which the shell handles for its own purposes.

The masks are easier to read after decoding one by hand. Take SigCgt, 0000000000010200.

1. Drop the leading zeros: 0x10200.
2. Write each hexadecimal digit as 4 bits: 1 0000 0010 0000 0000.
3. Number the bits from the right, starting at 0. The bits that are 1 are bit 9 and bit 16.
4. Add 1 to each: signals 10 and 17, SIGUSR1 and SIGCHLD.

SigIgn, 0x2, is the single bit 1, signal 2, SIGINT. A mask of all zeros, as for SigPnd and SigBlk here, means that no signal is pending and none is blocked.

The design has direct consequences, and the measurements below confirm each of them.

• The pending mask has 1 bit per signal, so it cannot count. A signal that is sent again while its bit is set leaves the mask unchanged.
• A blocked signal stays pending for as long as it is blocked.
• A handler is an address inside the current program. exec replaces the program, so a handled signal returns to the default action. Ignore and default are independent of the program and are kept.
• SIGKILL and SIGSTOP are exempt from all 3 mechanisms. The kernel guarantees that an administrator can always end or stop a process.

How a handler is run

A handler is a function in the program, and the kernel has to make the program call it at a moment the program did not plan for. It does so by editing the state it is about to return to.

1. The process is in the kernel, about to return to user mode, with its user-mode registers saved.
2. The kernel copies those saved registers to the user stack of the process, in a signal frame.
3. It changes the saved program counter to the address of the handler, and arranges the return address of the handler to point at a small piece of code that makes the system call sigreturn.
4. It returns to user mode. The process now executes the handler, as if the handler had been called at the point of interruption.
5. When the handler returns, sigreturn enters the kernel, which restores the registers from the signal frame. The program continues at the instruction where it was interrupted.

The handler can run between any 2 instructions of the program. If the program was in the middle of malloc, with the allocator’s data structures half updated, and the handler calls malloc too, the structures are corrupted. A handler may therefore call only functions that are async-signal-safe, a short list that includes write and excludes malloc and printf. The usual practice is a handler that does almost nothing: it sets a flag, or writes a byte to a pipe that the main loop of the program reads.

The runtimes of both languages do this for us. The Go runtime installs handlers for all signals and forwards them to the program through a channel registered with signal.Notify. Rust libraries such as signal-hook use the pipe technique. In both, the code we write for a signal runs in ordinary program context and may call anything.

Measuring signals

A program that changed its own signal handling would measure its language runtime as much as the kernel. sigfacts therefore observes other processes. It starts shell scripts that set dispositions with trap, sends them signals, and reads their masks from /proc/<pid>/status. The Rust version uses the nix crate with the signal feature.

$ cargo new sigfacts
$ cargo add -p sigfacts nix --features signal

mask reads a mask from the status file as a hexadecimal number. await waits until a process has reached a given state.

Listing 5.1 The imports of sigfacts, in Go. code/go/sigfacts/main.go

import (
	"bufio"
	"fmt"
	"io"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"syscall"
	"time"
)

Listing 5.2 The imports of sigfacts, in Rust. code/rust/sigfacts/src/main.rs

use std::fs;
use std::io::{self, Read};
use std::os::unix::process::ExitStatusExt;
use std::process::{Child, Command, Stdio};
use std::thread;
use std::time::Duration;

use nix::sys::signal::{Signal, kill};
use nix::unistd::Pid;

Listing 5.3 Reading the signal masks of another process, in Go. code/go/sigfacts/main.go

// mask returns one of the signal masks in /proc/<pid>/status: ShdPnd
// (pending), SigBlk (blocked), SigIgn (ignored) or SigCgt (caught).
func mask(pid int, name string) (uint64, error) {
	status, err := os.Open(fmt.Sprintf("/proc/%d/status", pid))

	if err != nil {
		return 0, err
	}

	defer status.Close()

	for lines := bufio.NewScanner(status); lines.Scan(); {
		if value, ok := strings.CutPrefix(lines.Text(), name+":"); ok {
			return strconv.ParseUint(strings.TrimSpace(value), 16, 64)
		}
	}

	return 0, fmt.Errorf("%s: not in /proc/%d/status", name, pid)
}

// has reports whether a mask contains a signal. Signal n is bit n-1.
func has(mask uint64, signal syscall.Signal) bool {
	return mask&(1<<(signal-1)) != 0
}

// await polls /proc/<pid>/stat until the process has the given command name
// and state. State T is "stopped" and S is "sleeping".
func await(pid int, command string, state byte) error {
	want := fmt.Sprintf("(%s) %c", command, state)

	for {
		stat, err := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))

		if err != nil {
			return err
		}

		if strings.Contains(string(stat), want) {
			return nil
		}

		time.Sleep(time.Millisecond)
	}
}

Listing 5.4 Reading the signal masks of another process, in Rust. code/rust/sigfacts/src/main.rs

/// Returns one of the signal masks in `/proc/<pid>/status`: `ShdPnd`
/// (pending), `SigBlk` (blocked), `SigIgn` (ignored) or `SigCgt` (caught).
fn mask(pid: Pid, name: &str) -> io::Result<u64> {
    let status = fs::read_to_string(format!("/proc/{pid}/status"))?;

    status
        .lines()
        .find_map(|line| line.strip_prefix(name)?.strip_prefix(':'))
        .and_then(|value| u64::from_str_radix(value.trim(), 16).ok())
        .ok_or_else(|| io::ErrorKind::InvalidData.into())
}

/// Reports whether a mask contains a signal. Signal `n` is bit `n - 1`.
fn has(mask: u64, signal: Signal) -> bool {
    mask & (1 << (signal as i32 - 1)) != 0
}

/// Polls `/proc/<pid>/stat` until the process has the given command name and
/// state. State `T` is "stopped" and `S` is "sleeping".
fn wait_for(pid: Pid, command: &str, state: char) -> io::Result<()> {
    let want = format!("({command}) {state}");

    while !fs::read_to_string(format!("/proc/{pid}/stat"))?.contains(&want) {
        thread::sleep(Duration::from_millis(1));
    }

    Ok(())
}

/// Starts a shell script and returns the child with its process number.
fn shell(script: &str, stdout: Stdio) -> io::Result<(Child, Pid)> {
    let child = Command::new("sh")
        .args(["-c", script])
        .stdout(stdout)
        .spawn()?;
    let pid = Pid::from_raw(child.id() as i32);

    Ok((child, pid))
}

A stopped process does not return to user mode, so nothing is delivered to it. pending uses that to observe the pending mask. The script installs a handler for SIGUSR1 and stops itself. The program sends SIGUSR1 3 times, reads the mask, and lets the script continue with SIGCONT.

Listing 5.5 Sending a signal 3 times to a stopped process, in Go. code/go/sigfacts/main.go

// pending sends SIGUSR1 3 times to a stopped process that has a handler for
// it, reads the pending mask, and counts how often the handler runs after
// the process continues.
func pending() error {
	child := exec.Command("sh", "-c", `trap 'echo caught' USR1; kill -STOP $$; echo done`)
	stdout, err := child.StdoutPipe()

	if err != nil {
		return err
	}

	if err := child.Start(); err != nil {
		return err
	}

	pid := child.Process.Pid

	if err := await(pid, "sh", 'T'); err != nil {
		return err
	}

	for range 3 {
		syscall.Kill(pid, syscall.SIGUSR1)
	}

	waiting, err := mask(pid, "ShdPnd")

	if err != nil {
		return err
	}

	syscall.Kill(pid, syscall.SIGCONT)

	text, err := io.ReadAll(stdout)

	if err != nil {
		return err
	}

	fmt.Printf("SIGUSR1 sent 3 times to a stopped process: pending mask %#x\n", waiting)
	fmt.Printf("after SIGCONT the handler ran %d time(s)\n", strings.Count(string(text), "caught"))

	return child.Wait()
}

Listing 5.6 Sending a signal 3 times to a stopped process, in Rust. code/rust/sigfacts/src/main.rs

/// Sends `SIGUSR1` 3 times to a stopped process that has a handler for it,
/// reads the pending mask, and counts how often the handler runs after the
/// process continues.
fn pending() -> io::Result<()> {
    let script = "trap 'echo caught' USR1; kill -STOP $$; echo done";
    let (mut child, pid) = shell(script, Stdio::piped())?;

    wait_for(pid, "sh", 'T')?;

    for _ in 0..3 {
        kill(pid, Signal::SIGUSR1)?;
    }

    let waiting = mask(pid, "ShdPnd")?;

    kill(pid, Signal::SIGCONT)?;

    let mut text = String::new();

    if let Some(stdout) = child.stdout.as_mut() {
        stdout.read_to_string(&mut text)?;
    }

    child.wait()?;

    println!("SIGUSR1 sent 3 times to a stopped process: pending mask {waiting:#x}");
    println!(
        "after SIGCONT the handler ran {} time(s)",
        text.matches("caught").count()
    );

    Ok(())
}

dispositions reads the ignored and caught masks of a shell that ignores SIGINT and handles SIGUSR1.

Listing 5.7 The actions of a process, read from outside, in Go. code/go/sigfacts/main.go

// dispositions reads the ignored and caught masks of a shell that ignores
// SIGINT and handles SIGUSR1.
func dispositions() error {
	child := exec.Command("sh", "-c", `trap '' INT; trap 'echo caught' USR1; kill -STOP $$`)

	if err := child.Start(); err != nil {
		return err
	}

	defer child.Wait()
	defer child.Process.Kill()

	pid := child.Process.Pid

	if err := await(pid, "sh", 'T'); err != nil {
		return err
	}

	ignored, err := mask(pid, "SigIgn")

	if err != nil {
		return err
	}

	caught, err := mask(pid, "SigCgt")

	if err != nil {
		return err
	}

	fmt.Printf("shell with 2 traps: SIGINT ignored %v, SIGUSR1 caught %v, SIGTERM ignored %v\n",
		has(ignored, syscall.SIGINT), has(caught, syscall.SIGUSR1), has(ignored, syscall.SIGTERM))

	return nil
}

Listing 5.8 The actions of a process, read from outside, in Rust. code/rust/sigfacts/src/main.rs

/// Reads the ignored and caught masks of a shell that ignores `SIGINT` and
/// handles `SIGUSR1`.
fn dispositions() -> io::Result<()> {
    let script = "trap '' INT; trap 'echo caught' USR1; kill -STOP $$";
    let (mut child, pid) = shell(script, Stdio::inherit())?;

    wait_for(pid, "sh", 'T')?;

    let ignored = mask(pid, "SigIgn")?;
    let caught = mask(pid, "SigCgt")?;

    child.kill()?;
    child.wait()?;

    println!(
        "shell with 2 traps: SIGINT ignored {}, SIGUSR1 caught {}, SIGTERM ignored {}",
        has(ignored, Signal::SIGINT),
        has(caught, Signal::SIGUSR1),
        has(ignored, Signal::SIGTERM)
    );

    Ok(())
}

acrossExec starts a shell that ignores SIGTERM, handles SIGUSR1, and then replaces itself with sleep by exec. It reads the masks of the sleep process, sends it SIGTERM and then SIGKILL, and reports how it ended.

Listing 5.9 Dispositions after exec, and SIGKILL, in Go. code/go/sigfacts/main.go

// acrossExec reads the masks of a process after exec, sends it a signal it
// ignores, and then SIGKILL.
func acrossExec() error {
	child := exec.Command("sh", "-c", `trap '' TERM; trap 'echo caught' USR1; exec sleep 30`)

	if err := child.Start(); err != nil {
		return err
	}

	pid := child.Process.Pid

	if err := await(pid, "sleep", 'S'); err != nil {
		return err
	}

	ignored, err := mask(pid, "SigIgn")

	if err != nil {
		return err
	}

	caught, err := mask(pid, "SigCgt")

	if err != nil {
		return err
	}

	fmt.Printf("after exec: SIGTERM ignored %v, SIGUSR1 caught %v\n",
		has(ignored, syscall.SIGTERM), has(caught, syscall.SIGUSR1))

	syscall.Kill(pid, syscall.SIGTERM)
	syscall.Kill(pid, syscall.SIGKILL)
	child.Wait()

	if status, ok := child.ProcessState.Sys().(syscall.WaitStatus); ok && status.Signaled() {
		fmt.Printf("SIGTERM, then SIGKILL: killed by signal %d\n", status.Signal())
	}

	return nil
}

Listing 5.10 Dispositions after exec, and SIGKILL, in Rust. code/rust/sigfacts/src/main.rs

/// Reads the masks of a process after exec, sends it a signal it ignores, and
/// then `SIGKILL`.
fn across_exec() -> io::Result<()> {
    let script = "trap '' TERM; trap 'echo caught' USR1; exec sleep 30";
    let (mut child, pid) = shell(script, Stdio::inherit())?;

    wait_for(pid, "sleep", 'S')?;

    let ignored = mask(pid, "SigIgn")?;
    let caught = mask(pid, "SigCgt")?;

    println!(
        "after exec: SIGTERM ignored {}, SIGUSR1 caught {}",
        has(ignored, Signal::SIGTERM),
        has(caught, Signal::SIGUSR1)
    );

    kill(pid, Signal::SIGTERM)?;
    kill(pid, Signal::SIGKILL)?;

    if let Some(signal) = child.wait()?.signal() {
        println!("SIGTERM, then SIGKILL: killed by signal {signal}");
    }

    Ok(())
}

SIGUSR1 sent 3 times to a stopped process: pending mask 0x200
after SIGCONT the handler ran 1 time(s)
shell with 2 traps: SIGINT ignored true, SIGUSR1 caught true, SIGTERM ignored false
after exec: SIGTERM ignored true, SIGUSR1 caught false
SIGTERM, then SIGKILL: killed by signal 9

Line 4 explains a detail from chapter 1. A Rust program starts with SIGPIPE ignored, and an ignored signal stays ignored across exec. Rust’s Command therefore resets SIGPIPE to the default in each child before it calls exec, and yes in the pipeline was ended by the signal as intended.

The merging of signals has a consequence for every program that handles SIGCHLD. A handler for it calls wait in a loop until no ended child remains, because several children may have ended behind a single delivery.

Modelling signal state

The state of a process is the 2 masks and a table of actions.

Listing 5.11 The imports of sigmodel, in Go. code/go/sigmodel/main.go

import (
	"errors"
	"fmt"
	"log"
)

Listing 5.12 The imports of sigmodel, in Rust. code/rust/sigmodel/src/main.rs

use std::fmt;

Listing 5.13 The signal state of a process, in Go. code/go/sigmodel/main.go

const (
	sigKill = 9
	sigUsr1 = 10
	sigTerm = 15
	sigChld = 17
	sigLast = 64
)

type action int

const (
	byDefault action = iota
	ignore
	handle
)

var errInvalid = errors.New("EINVAL")

// A process holds the signal state of one process. Signal n is bit n-1 of
// the masks.
type process struct {
	pending  uint64
	blocked  uint64
	actions  [sigLast + 1]action
	restart  [sigLast + 1]bool
	handled  int
	killedBy int

	// call is the system call the process is blocked in, or "" when it is
	// running. outcome records what a signal did to that call.
	call    string
	outcome string
}

func bit(signal int) uint64 {
	return 1 << (signal - 1)
}

Listing 5.14 The signal state of a process, in Rust. code/rust/sigmodel/src/main.rs

const SIGKILL: usize = 9;
const SIGUSR1: usize = 10;
const SIGTERM: usize = 15;
const SIGCHLD: usize = 17;
const SIGLAST: usize = 64;

#[derive(Clone, Copy, PartialEq)]
enum Action {
    Default,
    Ignore,
    Handle,
}

#[derive(Debug)]
struct Invalid;

impl fmt::Display for Invalid {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("EINVAL")
    }
}

/// The signal state of one process. Signal `n` is bit `n - 1` of the masks.
struct Process {
    pending: u64,
    blocked: u64,
    actions: [Action; SIGLAST + 1],
    restart: [bool; SIGLAST + 1],
    handled: u32,
    killed_by: Option<usize>,

    /// The system call the process is blocked in, or `None` when it is
    /// running. `outcome` records what a signal did to that call.
    call: Option<&'static str>,
    outcome: String,
}

fn bit(signal: usize) -> u64 {
    1 << (signal - 1)
}

impl Process {
    fn new(call: Option<&'static str>) -> Process {
        Process {
            pending: 0,
            blocked: 0,
            actions: [Action::Default; SIGLAST + 1],
            restart: [false; SIGLAST + 1],
            handled: 0,
            killed_by: None,
            call,
            outcome: String::new(),
        }
    }
}

setAction and block enforce the exemption of SIGKILL. Changing its action is an error, and a request to block it is reduced silently.

Listing 5.15 Installing actions and changing the blocked mask, in Go. code/go/sigmodel/main.go

// setAction installs an action for a signal. restart is the SA_RESTART flag
// of a handler. The action of SIGKILL cannot be changed.
func (p *process) setAction(signal int, a action, restart bool) error {
	if signal == sigKill {
		return errInvalid
	}

	p.actions[signal], p.restart[signal] = a, restart

	return nil
}

// block adds signals to the blocked mask. SIGKILL is removed from the
// request without an error, as the kernel does.
func (p *process) block(mask uint64) {
	p.blocked |= mask &^ bit(sigKill)
}

// unblock removes signals from the blocked mask and delivers what was
// waiting for it.
func (p *process) unblock(mask uint64) {
	p.blocked &^= mask
	p.deliver()
}

Listing 5.16 Installing actions and changing the blocked mask, in Rust. code/rust/sigmodel/src/main.rs

impl Process {
    /// Installs an action for a signal. `restart` is the `SA_RESTART` flag of
    /// a handler. The action of `SIGKILL` cannot be changed.
    fn set_action(&mut self, signal: usize, action: Action, restart: bool) -> Result<(), Invalid> {
        if signal == SIGKILL {
            return Err(Invalid);
        }

        self.actions[signal] = action;
        self.restart[signal] = restart;

        Ok(())
    }

    /// Adds signals to the blocked mask. `SIGKILL` is removed from the
    /// request without an error, as the kernel does.
    fn block(&mut self, mask: u64) {
        self.blocked |= mask & !bit(SIGKILL);
    }

    /// Removes signals from the blocked mask and delivers what was waiting
    /// for it.
    fn unblock(&mut self, mask: u64) {
        self.blocked &= !mask;
        self.deliver();
    }
}

send is the half that runs in the sender. A signal that the target ignores is discarded at this point and does not become pending, unless it is blocked: a blocked signal is kept, because its action may have changed by the time it is unblocked. Setting a bit that is already set has no effect, which is the merging we measured.

Listing 5.17 Sending: setting a pending bit, in Go. code/go/sigmodel/main.go

// ignored reports whether a signal would be discarded on delivery: its
// action is ignore, or its action is the default and the default is to
// ignore, which is the case for SIGCHLD.
func (p *process) ignored(signal int) bool {
	return p.actions[signal] == ignore || p.actions[signal] == byDefault && signal == sigChld
}

// send is the sender's half. It discards a signal that the target ignores
// and does not block. Otherwise it sets the pending bit, which is already
// set when the same signal is waiting.
func (p *process) send(signal int) {
	if p.ignored(signal) && p.blocked&bit(signal) == 0 {
		return
	}

	p.pending |= bit(signal)
}

Listing 5.18 Sending: setting a pending bit, in Rust. code/rust/sigmodel/src/main.rs

impl Process {
    /// Reports whether a signal would be discarded on delivery: its action is
    /// ignore, or its action is the default and the default is to ignore,
    /// which is the case for `SIGCHLD`.
    fn ignored(&self, signal: usize) -> bool {
        match self.actions[signal] {
            Action::Ignore => true,
            Action::Default => signal == SIGCHLD,
            Action::Handle => false,
        }
    }

    /// The sender's half. It discards a signal that the target ignores and
    /// does not block. Otherwise it sets the pending bit, which is already
    /// set when the same signal is waiting.
    fn send(&mut self, signal: usize) {
        if self.ignored(signal) && self.blocked & bit(signal) == 0 {
            return;
        }

        self.pending |= bit(signal);
    }
}

deliver is the half that runs in the target. It considers the signals that are pending and not blocked. For a handler it also decides the fate of a system call that the process was blocked in.

Listing 5.19 Delivering: performing the action, in Go. code/go/sigmodel/main.go

// deliver is the target's half. The kernel runs it when the process is about
// to return to user mode. It takes each pending signal that is not blocked,
// lowest number first, and performs its action.
func (p *process) deliver() {
	for signal := 1; signal <= sigLast && p.killedBy == 0; signal++ {
		if p.pending&^p.blocked&bit(signal) == 0 {
			continue
		}

		p.pending &^= bit(signal)

		switch {
		case p.actions[signal] == handle:
			p.handled++
			p.interrupt(signal)

		case !p.ignored(signal):
			p.killedBy = signal
		}
	}
}

// interrupt decides what happens to a blocking system call when a handler
// has run: the call is restarted, or it fails with EINTR.
func (p *process) interrupt(signal int) {
	if p.call == "" {
		return
	}

	if p.restart[signal] {
		p.outcome = "the " + p.call + " is restarted"

		return
	}

	p.outcome = "the " + p.call + " fails with EINTR"
	p.call = ""
}

Listing 5.20 Delivering: performing the action, in Rust. code/rust/sigmodel/src/main.rs

impl Process {
    /// The target's half. The kernel runs it when the process is about to
    /// return to user mode. It takes each pending signal that is not blocked,
    /// lowest number first, and performs its action.
    fn deliver(&mut self) {
        for signal in 1..=SIGLAST {
            if self.killed_by.is_some() {
                break;
            }

            if self.pending & !self.blocked & bit(signal) == 0 {
                continue;
            }

            self.pending &= !bit(signal);

            if self.actions[signal] == Action::Handle {
                self.handled += 1;
                self.interrupt(signal);
            } else if !self.ignored(signal) {
                self.killed_by = Some(signal);
            }
        }
    }

    /// Decides what happens to a blocking system call when a handler has run:
    /// the call is restarted, or it fails with `EINTR`.
    fn interrupt(&mut self, signal: usize) {
        let Some(call) = self.call else {
            return;
        };

        if self.restart[signal] {
            self.outcome = format!("the {call} is restarted");

            return;
        }

        self.outcome = format!("the {call} fails with EINTR");
        self.call = None;
    }
}

A signal during a blocking call

A process that is blocked in a system call is inside the kernel and is not about to return to user mode. If signals waited for the call to finish, a process blocked in a read on a terminal could not be interrupted by Ctrl-C. The kernel therefore wakes the process, abandons the call, and runs the handler. After the handler returns there are 2 possible outcomes, selected by a flag given to sigaction when the handler was installed.

Without SA_RESTART, the call returns the error EINTR, “interrupted system call”, and the program has to call it again. A loop that retries on EINTR around every blocking call is the standard defence. Go’s standard library and Rust’s std::io helpers such as read_to_end contain that loop.

Listing 5.21 The trace, in Go. code/go/sigmodel/main.go

func coalescing() error {
	p := &process{}

	if err := p.setAction(sigUsr1, handle, false); err != nil {
		return err
	}

	p.block(bit(sigUsr1))

	for range 3 {
		p.send(sigUsr1)
		p.deliver()
	}

	fmt.Printf("SIGUSR1 blocked and sent 3 times: pending mask %#x\n", p.pending)

	p.unblock(bit(sigUsr1))
	fmt.Printf("after unblocking the handler ran %d time(s)\n", p.handled)

	return nil
}

func ignoring() error {
	p := &process{}

	if err := p.setAction(sigTerm, ignore, false); err != nil {
		return err
	}

	p.send(sigTerm)
	fmt.Printf("SIGTERM ignored and sent: pending mask %#x\n", p.pending)

	fmt.Printf("installing a handler for SIGKILL: %v\n", p.setAction(sigKill, handle, false))

	p.block(^uint64(0))
	p.send(sigKill)
	p.deliver()
	fmt.Printf("every signal blocked, SIGKILL sent: killed by signal %d\n", p.killedBy)

	return nil
}

func interrupting(restart bool) error {
	p := &process{call: "read"}

	if err := p.setAction(sigUsr1, handle, restart); err != nil {
		return err
	}

	p.send(sigUsr1)
	p.deliver()
	fmt.Printf("handler runs during a blocking read, SA_RESTART %v: %s\n", restart, p.outcome)

	return nil
}

func main() {
	steps := []func() error{
		coalescing,
		ignoring,
		func() error { return interrupting(false) },
		func() error { return interrupting(true) },
	}

	for _, step := range steps {
		if err := step(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 5.22 The trace, in Rust. code/rust/sigmodel/src/main.rs

fn coalescing() -> Result<(), Invalid> {
    let mut process = Process::new(None);

    process.set_action(SIGUSR1, Action::Handle, false)?;
    process.block(bit(SIGUSR1));

    for _ in 0..3 {
        process.send(SIGUSR1);
        process.deliver();
    }

    println!(
        "SIGUSR1 blocked and sent 3 times: pending mask {:#x}",
        process.pending
    );

    process.unblock(bit(SIGUSR1));
    println!(
        "after unblocking the handler ran {} time(s)",
        process.handled
    );

    Ok(())
}

fn ignoring() -> Result<(), Invalid> {
    let mut process = Process::new(None);

    process.set_action(SIGTERM, Action::Ignore, false)?;
    process.send(SIGTERM);
    println!(
        "SIGTERM ignored and sent: pending mask {:#x}",
        process.pending
    );

    if let Err(error) = process.set_action(SIGKILL, Action::Handle, false) {
        println!("installing a handler for SIGKILL: {error}");
    }

    process.block(u64::MAX);
    process.send(SIGKILL);
    process.deliver();

    if let Some(signal) = process.killed_by {
        println!("every signal blocked, SIGKILL sent: killed by signal {signal}");
    }

    Ok(())
}

fn interrupting(restart: bool) -> Result<(), Invalid> {
    let mut process = Process::new(Some("read"));

    process.set_action(SIGUSR1, Action::Handle, restart)?;
    process.send(SIGUSR1);
    process.deliver();

    println!(
        "handler runs during a blocking read, SA_RESTART {restart}: {}",
        process.outcome
    );

    Ok(())
}

fn main() -> Result<(), Invalid> {
    coalescing()?;
    ignoring()?;
    interrupting(false)?;
    interrupting(true)
}

SIGUSR1 blocked and sent 3 times: pending mask 0x200
after unblocking the handler ran 1 time(s)
SIGTERM ignored and sent: pending mask 0x0
installing a handler for SIGKILL: EINVAL
every signal blocked, SIGKILL sent: killed by signal 9
handler runs during a blocking read, SA_RESTART false: the read fails with EINTR
handler runs during a blocking read, SA_RESTART true: the read is restarted

Lines 1 and 2 reproduce the pending mask and the single delivery that sigfacts measured.

Inspecting signals on a live system

A process that does not react to kill is diagnosed with the second command. If SIGTERM, bit 14, is set in SigIgn or SigBlk, the process has ignored or blocked it, and only SIGKILL will end it.

The complete programs

Listing 5.23 The complete sigfacts, in Go. code/go/sigfacts/main.go

// Command sigfacts measures signal handling from outside a process. It starts
// shell scripts that set signal dispositions, sends them signals, and reads
// their signal masks from /proc/<pid>/status.
//
// Run it from code/go with:
//
//	go run ./sigfacts
package main

import (
	"bufio"
	"fmt"
	"io"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"syscall"
	"time"
)


// mask returns one of the signal masks in /proc/<pid>/status: ShdPnd
// (pending), SigBlk (blocked), SigIgn (ignored) or SigCgt (caught).
func mask(pid int, name string) (uint64, error) {
	status, err := os.Open(fmt.Sprintf("/proc/%d/status", pid))

	if err != nil {
		return 0, err
	}

	defer status.Close()

	for lines := bufio.NewScanner(status); lines.Scan(); {
		if value, ok := strings.CutPrefix(lines.Text(), name+":"); ok {
			return strconv.ParseUint(strings.TrimSpace(value), 16, 64)
		}
	}

	return 0, fmt.Errorf("%s: not in /proc/%d/status", name, pid)
}

// has reports whether a mask contains a signal. Signal n is bit n-1.
func has(mask uint64, signal syscall.Signal) bool {
	return mask&(1<<(signal-1)) != 0
}

// await polls /proc/<pid>/stat until the process has the given command name
// and state. State T is "stopped" and S is "sleeping".
func await(pid int, command string, state byte) error {
	want := fmt.Sprintf("(%s) %c", command, state)

	for {
		stat, err := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))

		if err != nil {
			return err
		}

		if strings.Contains(string(stat), want) {
			return nil
		}

		time.Sleep(time.Millisecond)
	}
}


// pending sends SIGUSR1 3 times to a stopped process that has a handler for
// it, reads the pending mask, and counts how often the handler runs after
// the process continues.
func pending() error {
	child := exec.Command("sh", "-c", `trap 'echo caught' USR1; kill -STOP $$; echo done`)
	stdout, err := child.StdoutPipe()

	if err != nil {
		return err
	}

	if err := child.Start(); err != nil {
		return err
	}

	pid := child.Process.Pid

	if err := await(pid, "sh", 'T'); err != nil {
		return err
	}

	for range 3 {
		syscall.Kill(pid, syscall.SIGUSR1)
	}

	waiting, err := mask(pid, "ShdPnd")

	if err != nil {
		return err
	}

	syscall.Kill(pid, syscall.SIGCONT)

	text, err := io.ReadAll(stdout)

	if err != nil {
		return err
	}

	fmt.Printf("SIGUSR1 sent 3 times to a stopped process: pending mask %#x\n", waiting)
	fmt.Printf("after SIGCONT the handler ran %d time(s)\n", strings.Count(string(text), "caught"))

	return child.Wait()
}


// dispositions reads the ignored and caught masks of a shell that ignores
// SIGINT and handles SIGUSR1.
func dispositions() error {
	child := exec.Command("sh", "-c", `trap '' INT; trap 'echo caught' USR1; kill -STOP $$`)

	if err := child.Start(); err != nil {
		return err
	}

	defer child.Wait()
	defer child.Process.Kill()

	pid := child.Process.Pid

	if err := await(pid, "sh", 'T'); err != nil {
		return err
	}

	ignored, err := mask(pid, "SigIgn")

	if err != nil {
		return err
	}

	caught, err := mask(pid, "SigCgt")

	if err != nil {
		return err
	}

	fmt.Printf("shell with 2 traps: SIGINT ignored %v, SIGUSR1 caught %v, SIGTERM ignored %v\n",
		has(ignored, syscall.SIGINT), has(caught, syscall.SIGUSR1), has(ignored, syscall.SIGTERM))

	return nil
}


// acrossExec reads the masks of a process after exec, sends it a signal it
// ignores, and then SIGKILL.
func acrossExec() error {
	child := exec.Command("sh", "-c", `trap '' TERM; trap 'echo caught' USR1; exec sleep 30`)

	if err := child.Start(); err != nil {
		return err
	}

	pid := child.Process.Pid

	if err := await(pid, "sleep", 'S'); err != nil {
		return err
	}

	ignored, err := mask(pid, "SigIgn")

	if err != nil {
		return err
	}

	caught, err := mask(pid, "SigCgt")

	if err != nil {
		return err
	}

	fmt.Printf("after exec: SIGTERM ignored %v, SIGUSR1 caught %v\n",
		has(ignored, syscall.SIGTERM), has(caught, syscall.SIGUSR1))

	syscall.Kill(pid, syscall.SIGTERM)
	syscall.Kill(pid, syscall.SIGKILL)
	child.Wait()

	if status, ok := child.ProcessState.Sys().(syscall.WaitStatus); ok && status.Signaled() {
		fmt.Printf("SIGTERM, then SIGKILL: killed by signal %d\n", status.Signal())
	}

	return nil
}


func main() {
	for _, experiment := range []func() error{pending, dispositions, acrossExec} {
		if err := experiment(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 5.24 The complete sigfacts, in Rust. code/rust/sigfacts/src/main.rs

//! Measures signal handling from outside a process. It starts shell scripts
//! that set signal dispositions, sends them signals, and reads their signal
//! masks from `/proc/<pid>/status`.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p sigfacts
//! ```

use std::fs;
use std::io::{self, Read};
use std::os::unix::process::ExitStatusExt;
use std::process::{Child, Command, Stdio};
use std::thread;
use std::time::Duration;

use nix::sys::signal::{Signal, kill};
use nix::unistd::Pid;

/// Returns one of the signal masks in `/proc/<pid>/status`: `ShdPnd`
/// (pending), `SigBlk` (blocked), `SigIgn` (ignored) or `SigCgt` (caught).
fn mask(pid: Pid, name: &str) -> io::Result<u64> {
    let status = fs::read_to_string(format!("/proc/{pid}/status"))?;

    status
        .lines()
        .find_map(|line| line.strip_prefix(name)?.strip_prefix(':'))
        .and_then(|value| u64::from_str_radix(value.trim(), 16).ok())
        .ok_or_else(|| io::ErrorKind::InvalidData.into())
}

/// Reports whether a mask contains a signal. Signal `n` is bit `n - 1`.
fn has(mask: u64, signal: Signal) -> bool {
    mask & (1 << (signal as i32 - 1)) != 0
}

/// Polls `/proc/<pid>/stat` until the process has the given command name and
/// state. State `T` is "stopped" and `S` is "sleeping".
fn wait_for(pid: Pid, command: &str, state: char) -> io::Result<()> {
    let want = format!("({command}) {state}");

    while !fs::read_to_string(format!("/proc/{pid}/stat"))?.contains(&want) {
        thread::sleep(Duration::from_millis(1));
    }

    Ok(())
}

/// Starts a shell script and returns the child with its process number.
fn shell(script: &str, stdout: Stdio) -> io::Result<(Child, Pid)> {
    let child = Command::new("sh")
        .args(["-c", script])
        .stdout(stdout)
        .spawn()?;
    let pid = Pid::from_raw(child.id() as i32);

    Ok((child, pid))
}

/// Sends `SIGUSR1` 3 times to a stopped process that has a handler for it,
/// reads the pending mask, and counts how often the handler runs after the
/// process continues.
fn pending() -> io::Result<()> {
    let script = "trap 'echo caught' USR1; kill -STOP $$; echo done";
    let (mut child, pid) = shell(script, Stdio::piped())?;

    wait_for(pid, "sh", 'T')?;

    for _ in 0..3 {
        kill(pid, Signal::SIGUSR1)?;
    }

    let waiting = mask(pid, "ShdPnd")?;

    kill(pid, Signal::SIGCONT)?;

    let mut text = String::new();

    if let Some(stdout) = child.stdout.as_mut() {
        stdout.read_to_string(&mut text)?;
    }

    child.wait()?;

    println!("SIGUSR1 sent 3 times to a stopped process: pending mask {waiting:#x}");
    println!(
        "after SIGCONT the handler ran {} time(s)",
        text.matches("caught").count()
    );

    Ok(())
}

/// Reads the ignored and caught masks of a shell that ignores `SIGINT` and
/// handles `SIGUSR1`.
fn dispositions() -> io::Result<()> {
    let script = "trap '' INT; trap 'echo caught' USR1; kill -STOP $$";
    let (mut child, pid) = shell(script, Stdio::inherit())?;

    wait_for(pid, "sh", 'T')?;

    let ignored = mask(pid, "SigIgn")?;
    let caught = mask(pid, "SigCgt")?;

    child.kill()?;
    child.wait()?;

    println!(
        "shell with 2 traps: SIGINT ignored {}, SIGUSR1 caught {}, SIGTERM ignored {}",
        has(ignored, Signal::SIGINT),
        has(caught, Signal::SIGUSR1),
        has(ignored, Signal::SIGTERM)
    );

    Ok(())
}

/// Reads the masks of a process after exec, sends it a signal it ignores, and
/// then `SIGKILL`.
fn across_exec() -> io::Result<()> {
    let script = "trap '' TERM; trap 'echo caught' USR1; exec sleep 30";
    let (mut child, pid) = shell(script, Stdio::inherit())?;

    wait_for(pid, "sleep", 'S')?;

    let ignored = mask(pid, "SigIgn")?;
    let caught = mask(pid, "SigCgt")?;

    println!(
        "after exec: SIGTERM ignored {}, SIGUSR1 caught {}",
        has(ignored, Signal::SIGTERM),
        has(caught, Signal::SIGUSR1)
    );

    kill(pid, Signal::SIGTERM)?;
    kill(pid, Signal::SIGKILL)?;

    if let Some(signal) = child.wait()?.signal() {
        println!("SIGTERM, then SIGKILL: killed by signal {signal}");
    }

    Ok(())
}

fn main() -> io::Result<()> {
    pending()?;
    dispositions()?;
    across_exec()
}

Listing 5.25 The complete sigmodel, in Go. code/go/sigmodel/main.go

// Command sigmodel models the signal state the kernel keeps for a process: a
// pending mask, a blocked mask, and an action for each signal. It separates
// sending a signal from delivering it, and shows what a delivered signal
// does to a process that is blocked in a system call.
//
// Run it from code/go with:
//
//	go run ./sigmodel
package main

import (
	"errors"
	"fmt"
	"log"
)


const (
	sigKill = 9
	sigUsr1 = 10
	sigTerm = 15
	sigChld = 17
	sigLast = 64
)

type action int

const (
	byDefault action = iota
	ignore
	handle
)

var errInvalid = errors.New("EINVAL")

// A process holds the signal state of one process. Signal n is bit n-1 of
// the masks.
type process struct {
	pending  uint64
	blocked  uint64
	actions  [sigLast + 1]action
	restart  [sigLast + 1]bool
	handled  int
	killedBy int

	// call is the system call the process is blocked in, or "" when it is
	// running. outcome records what a signal did to that call.
	call    string
	outcome string
}

func bit(signal int) uint64 {
	return 1 << (signal - 1)
}


// setAction installs an action for a signal. restart is the SA_RESTART flag
// of a handler. The action of SIGKILL cannot be changed.
func (p *process) setAction(signal int, a action, restart bool) error {
	if signal == sigKill {
		return errInvalid
	}

	p.actions[signal], p.restart[signal] = a, restart

	return nil
}

// block adds signals to the blocked mask. SIGKILL is removed from the
// request without an error, as the kernel does.
func (p *process) block(mask uint64) {
	p.blocked |= mask &^ bit(sigKill)
}

// unblock removes signals from the blocked mask and delivers what was
// waiting for it.
func (p *process) unblock(mask uint64) {
	p.blocked &^= mask
	p.deliver()
}


// ignored reports whether a signal would be discarded on delivery: its
// action is ignore, or its action is the default and the default is to
// ignore, which is the case for SIGCHLD.
func (p *process) ignored(signal int) bool {
	return p.actions[signal] == ignore || p.actions[signal] == byDefault && signal == sigChld
}

// send is the sender's half. It discards a signal that the target ignores
// and does not block. Otherwise it sets the pending bit, which is already
// set when the same signal is waiting.
func (p *process) send(signal int) {
	if p.ignored(signal) && p.blocked&bit(signal) == 0 {
		return
	}

	p.pending |= bit(signal)
}


// deliver is the target's half. The kernel runs it when the process is about
// to return to user mode. It takes each pending signal that is not blocked,
// lowest number first, and performs its action.
func (p *process) deliver() {
	for signal := 1; signal <= sigLast && p.killedBy == 0; signal++ {
		if p.pending&^p.blocked&bit(signal) == 0 {
			continue
		}

		p.pending &^= bit(signal)

		switch {
		case p.actions[signal] == handle:
			p.handled++
			p.interrupt(signal)

		case !p.ignored(signal):
			p.killedBy = signal
		}
	}
}

// interrupt decides what happens to a blocking system call when a handler
// has run: the call is restarted, or it fails with EINTR.
func (p *process) interrupt(signal int) {
	if p.call == "" {
		return
	}

	if p.restart[signal] {
		p.outcome = "the " + p.call + " is restarted"

		return
	}

	p.outcome = "the " + p.call + " fails with EINTR"
	p.call = ""
}


func coalescing() error {
	p := &process{}

	if err := p.setAction(sigUsr1, handle, false); err != nil {
		return err
	}

	p.block(bit(sigUsr1))

	for range 3 {
		p.send(sigUsr1)
		p.deliver()
	}

	fmt.Printf("SIGUSR1 blocked and sent 3 times: pending mask %#x\n", p.pending)

	p.unblock(bit(sigUsr1))
	fmt.Printf("after unblocking the handler ran %d time(s)\n", p.handled)

	return nil
}

func ignoring() error {
	p := &process{}

	if err := p.setAction(sigTerm, ignore, false); err != nil {
		return err
	}

	p.send(sigTerm)
	fmt.Printf("SIGTERM ignored and sent: pending mask %#x\n", p.pending)

	fmt.Printf("installing a handler for SIGKILL: %v\n", p.setAction(sigKill, handle, false))

	p.block(^uint64(0))
	p.send(sigKill)
	p.deliver()
	fmt.Printf("every signal blocked, SIGKILL sent: killed by signal %d\n", p.killedBy)

	return nil
}

func interrupting(restart bool) error {
	p := &process{call: "read"}

	if err := p.setAction(sigUsr1, handle, restart); err != nil {
		return err
	}

	p.send(sigUsr1)
	p.deliver()
	fmt.Printf("handler runs during a blocking read, SA_RESTART %v: %s\n", restart, p.outcome)

	return nil
}

func main() {
	steps := []func() error{
		coalescing,
		ignoring,
		func() error { return interrupting(false) },
		func() error { return interrupting(true) },
	}

	for _, step := range steps {
		if err := step(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 5.26 The complete sigmodel, in Rust. code/rust/sigmodel/src/main.rs

//! Models the signal state the kernel keeps for a process: a pending mask, a
//! blocked mask, and an action for each signal. It separates sending a signal
//! from delivering it, and shows what a delivered signal does to a process
//! that is blocked in a system call.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p sigmodel
//! ```

use std::fmt;

const SIGKILL: usize = 9;
const SIGUSR1: usize = 10;
const SIGTERM: usize = 15;
const SIGCHLD: usize = 17;
const SIGLAST: usize = 64;

#[derive(Clone, Copy, PartialEq)]
enum Action {
    Default,
    Ignore,
    Handle,
}

#[derive(Debug)]
struct Invalid;

impl fmt::Display for Invalid {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("EINVAL")
    }
}

/// The signal state of one process. Signal `n` is bit `n - 1` of the masks.
struct Process {
    pending: u64,
    blocked: u64,
    actions: [Action; SIGLAST + 1],
    restart: [bool; SIGLAST + 1],
    handled: u32,
    killed_by: Option<usize>,

    /// The system call the process is blocked in, or `None` when it is
    /// running. `outcome` records what a signal did to that call.
    call: Option<&'static str>,
    outcome: String,
}

fn bit(signal: usize) -> u64 {
    1 << (signal - 1)
}

impl Process {
    fn new(call: Option<&'static str>) -> Process {
        Process {
            pending: 0,
            blocked: 0,
            actions: [Action::Default; SIGLAST + 1],
            restart: [false; SIGLAST + 1],
            handled: 0,
            killed_by: None,
            call,
            outcome: String::new(),
        }
    }
}

impl Process {
    /// Installs an action for a signal. `restart` is the `SA_RESTART` flag of
    /// a handler. The action of `SIGKILL` cannot be changed.
    fn set_action(&mut self, signal: usize, action: Action, restart: bool) -> Result<(), Invalid> {
        if signal == SIGKILL {
            return Err(Invalid);
        }

        self.actions[signal] = action;
        self.restart[signal] = restart;

        Ok(())
    }

    /// Adds signals to the blocked mask. `SIGKILL` is removed from the
    /// request without an error, as the kernel does.
    fn block(&mut self, mask: u64) {
        self.blocked |= mask & !bit(SIGKILL);
    }

    /// Removes signals from the blocked mask and delivers what was waiting
    /// for it.
    fn unblock(&mut self, mask: u64) {
        self.blocked &= !mask;
        self.deliver();
    }
}

impl Process {
    /// Reports whether a signal would be discarded on delivery: its action is
    /// ignore, or its action is the default and the default is to ignore,
    /// which is the case for `SIGCHLD`.
    fn ignored(&self, signal: usize) -> bool {
        match self.actions[signal] {
            Action::Ignore => true,
            Action::Default => signal == SIGCHLD,
            Action::Handle => false,
        }
    }

    /// The sender's half. It discards a signal that the target ignores and
    /// does not block. Otherwise it sets the pending bit, which is already
    /// set when the same signal is waiting.
    fn send(&mut self, signal: usize) {
        if self.ignored(signal) && self.blocked & bit(signal) == 0 {
            return;
        }

        self.pending |= bit(signal);
    }
}

impl Process {
    /// The target's half. The kernel runs it when the process is about to
    /// return to user mode. It takes each pending signal that is not blocked,
    /// lowest number first, and performs its action.
    fn deliver(&mut self) {
        for signal in 1..=SIGLAST {
            if self.killed_by.is_some() {
                break;
            }

            if self.pending & !self.blocked & bit(signal) == 0 {
                continue;
            }

            self.pending &= !bit(signal);

            if self.actions[signal] == Action::Handle {
                self.handled += 1;
                self.interrupt(signal);
            } else if !self.ignored(signal) {
                self.killed_by = Some(signal);
            }
        }
    }

    /// Decides what happens to a blocking system call when a handler has run:
    /// the call is restarted, or it fails with `EINTR`.
    fn interrupt(&mut self, signal: usize) {
        let Some(call) = self.call else {
            return;
        };

        if self.restart[signal] {
            self.outcome = format!("the {call} is restarted");

            return;
        }

        self.outcome = format!("the {call} fails with EINTR");
        self.call = None;
    }
}

fn coalescing() -> Result<(), Invalid> {
    let mut process = Process::new(None);

    process.set_action(SIGUSR1, Action::Handle, false)?;
    process.block(bit(SIGUSR1));

    for _ in 0..3 {
        process.send(SIGUSR1);
        process.deliver();
    }

    println!(
        "SIGUSR1 blocked and sent 3 times: pending mask {:#x}",
        process.pending
    );

    process.unblock(bit(SIGUSR1));
    println!(
        "after unblocking the handler ran {} time(s)",
        process.handled
    );

    Ok(())
}

fn ignoring() -> Result<(), Invalid> {
    let mut process = Process::new(None);

    process.set_action(SIGTERM, Action::Ignore, false)?;
    process.send(SIGTERM);
    println!(
        "SIGTERM ignored and sent: pending mask {:#x}",
        process.pending
    );

    if let Err(error) = process.set_action(SIGKILL, Action::Handle, false) {
        println!("installing a handler for SIGKILL: {error}");
    }

    process.block(u64::MAX);
    process.send(SIGKILL);
    process.deliver();

    if let Some(signal) = process.killed_by {
        println!("every signal blocked, SIGKILL sent: killed by signal {signal}");
    }

    Ok(())
}

fn interrupting(restart: bool) -> Result<(), Invalid> {
    let mut process = Process::new(Some("read"));

    process.set_action(SIGUSR1, Action::Handle, restart)?;
    process.send(SIGUSR1);
    process.deliver();

    println!(
        "handler runs during a blocking read, SA_RESTART {restart}: {}",
        process.outcome
    );

    Ok(())
}

fn main() -> Result<(), Invalid> {
    coalescing()?;
    ignoring()?;
    interrupting(false)?;
    interrupting(true)
}

References

• signal(7), sigaction(2), sigprocmask(2), signal-safety(7) and sigreturn(2), Linux manual pages: https://man7.org/linux/man-pages/man7/signal.7.html
• M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010, chapters 20 to 22.
• kernel/signal.c and include/linux/sched/signal.h, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/kernel/signal.c

CPU Scheduling

A machine has a fixed number of CPUs and usually many more tasks that want to run. The scheduler is the part of the kernel that decides, for each CPU and at each moment, which task runs. A task is the unit it schedules: a process with a single thread, or one thread of a process with several (chapter 3).

In this chapter, we will learn how the Completely Fair Scheduler, the default of Linux 5.15, makes that decision. We will first see how the scheduler gets control of a CPU at all, and follow a switch from one task to the next. Then we will measure how it divides a CPU between busy processes of different priorities with schedfacts, and write cfs, a model of its central rule that produces the measured shares.

When the scheduler runs

A task that is running holds its CPU, and the kernel is not executing. The scheduler can act only when control passes to the kernel. That happens in 3 ways.

The timer interrupt is what makes it impossible for a task to keep a CPU for ever. A program in an endless loop makes no system calls, and it is still interrupted, 250 times per second on our machine. Taking the CPU from a task that did not offer it is called preemption.

The kernel counts both kinds of switch for every task. This is a shell loop that had been running for a second on a busy machine:

$ grep ctxt /proc/577022/status
voluntary_ctxt_switches:	0
nonvoluntary_ctxt_switches:	37

The loop did not sleep once, and was preempted 37 times. A program that waits for input shows the opposite pattern.

From one task to the next

1. Task A is running on a CPU. The timer interrupt arrives, and the processor enters the kernel on A’s behalf.
2. The kernel charges A for the time it has run since it was last charged.
3. It checks whether A has had its share for now. If so, it sets a flag on A that asks for a reschedule.
4. On the way back to user mode, the kernel sees the flag and calls the scheduler.
5. The scheduler picks the next task, B, from the run queue: the set of tasks that are ready to run on this CPU.
6. The kernel performs a context switch. It saves the registers of A in A’s task_struct, loads the registers of B, and switches to B’s address space. A’s state remains complete, so it will continue later as if nothing had happened.
7. The return to user mode now resumes B. A waits in the run queue.

Steps 2, 3 and 5 are the scheduling policy: how much each task is charged, when it has had enough, and who is next. The rest of the chapter is about those 3 decisions.

Classes and policies

Linux has several schedulers side by side, called scheduling classes. Each task belongs to one, selected by its policy. When a CPU needs a task, the kernel asks the classes in a fixed order and takes the first task offered.

$ chrt -p 577022
pid 577022's current scheduling policy: SCHED_OTHER
pid 577022's current scheduling priority: 0

SCHED_OTHER is the policy of every task that has not asked for something else, and it belongs to the fair class. The real-time policies are for tasks that must run within a bounded time of becoming runnable, such as audio processing, and they require privilege, because a real-time task in an endless loop keeps its CPU from every ordinary task. The rest of this chapter concerns the fair class.

The requirement

The goal of the fair class is stated in terms of an ideal: if n tasks of equal priority are runnable on a CPU, each should receive 1/n of it, and a task of higher priority should receive a proportionally larger part. A real CPU runs a single task at a time, so the scheduler approximates the ideal by switching between tasks and keeping account of what each has received.

The priority of a task in the fair class is its nice value, an integer from -20 to 19. A higher value means a lower priority: the task is “nicer” to the others. The nice command and the setpriority system call set it, and only a privileged process may set a negative value. The kernel converts the nice value to a weight with a fixed table, in which nice 0 has weight 1024 and each step changes the weight by about 25%. The share of a task is its weight divided by the sum of the weights of all runnable tasks on the CPU.

A share is relative. A task of nice 19 that is alone on a CPU receives all of it. The nice value has an effect only when tasks compete.

Measuring shares

schedfacts starts 2 busy processes, restricts both to CPU 0 with taskset, and sets their nice values with nice. Restricting them to the same CPU makes them compete: on an idle machine with 8 CPUs each would otherwise get a CPU of its own. After 2 seconds the program reads the CPU time of each from /proc/<pid>/stat. It needs only the standard libraries.

$ cargo new schedfacts

spin starts a busy shell loop, and cpuTime reads the time a process has spent on a CPU. The kernel reports it in clock ticks of 10 ms, in separate fields for the time spent in user mode and in kernel mode.

Listing 6.1 The imports of schedfacts, in Go. code/go/schedfacts/main.go

import (
	"fmt"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"time"
)

Listing 6.2 The imports of schedfacts, in Rust. code/rust/schedfacts/src/main.rs

use std::fs;
use std::io;
use std::process::{Child, Command};
use std::thread;
use std::time::Duration;

Listing 6.3 Starting a busy process on CPU 0, and reading its CPU time, in Go. code/go/schedfacts/main.go

// spin starts a busy shell loop, restricted to CPU 0 and with
// the given nice value.
func spin(nice int) (*exec.Cmd, error) {
	child := exec.Command("taskset", "-c", "0", "nice", "-n", strconv.Itoa(nice), "sh", "-c", "while :; do :; done")

	return child, child.Start()
}

// cpuTime returns the CPU time a process has used, in clock ticks: fields 14
// and 15 of /proc/<pid>/stat, the time in user mode and in kernel mode.
func cpuTime(pid int) (int, error) {
	stat, err := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))

	if err != nil {
		return 0, err
	}

	fields := strings.Fields(string(stat[strings.LastIndexByte(string(stat), ')')+1:]))

	if len(fields) < 13 {
		return 0, fmt.Errorf("/proc/%d/stat: too few fields", pid)
	}

	var user, kernel int

	if _, err := fmt.Sscan(fields[11], &user); err != nil {
		return 0, err
	}

	if _, err := fmt.Sscan(fields[12], &kernel); err != nil {
		return 0, err
	}

	return user + kernel, nil
}

Listing 6.4 Starting a busy process on CPU 0, and reading its CPU time, in Rust. code/rust/schedfacts/src/main.rs

/// Starts a busy shell loop, restricted to CPU 0 and with the
/// given nice value.
fn spin(nice: i32) -> io::Result<Child> {
    Command::new("taskset")
        .args(["-c", "0", "nice", "-n", &nice.to_string()])
        .args(["sh", "-c", "while :; do :; done"])
        .spawn()
}

/// Returns the CPU time a process has used, in clock ticks: fields 14 and 15
/// of `/proc/<pid>/stat`, the time in user mode and in kernel mode.
fn cpu_time(pid: u32) -> io::Result<u64> {
    let stat = fs::read_to_string(format!("/proc/{pid}/stat"))?;
    let rest = stat.rsplit_once(')').map_or("", |(_, rest)| rest);

    let ticks: Vec<u64> = rest
        .split_whitespace()
        .skip(11)
        .take(2)
        .filter_map(|field| field.parse().ok())
        .collect();

    match ticks.as_slice() {
        [user, kernel] => Ok(user + kernel),
        _ => Err(io::ErrorKind::InvalidData.into()),
    }
}

share lets a process of nice 0 compete with a process of a given nice value, and prints the percentage each received. The percentages are rounded to the nearest 5, since a measurement over 2 seconds varies by a few ticks between runs.

Listing 6.5 Measuring the division of a CPU, in Go. code/go/schedfacts/main.go

// stop ends a busy process and collects it.
func stop(child *exec.Cmd) {
	child.Process.Kill()
	child.Wait()
}

// share lets a process of nice 0 and a process of the given nice value
// compete for CPU 0 for 2 seconds and prints the share of CPU time each
// received, rounded to the nearest 5%.
func share(nice int) error {
	first, err := spin(0)

	if err != nil {
		return err
	}

	defer stop(first)

	second, err := spin(nice)

	if err != nil {
		return err
	}

	defer stop(second)

	time.Sleep(2 * time.Second)

	a, err := cpuTime(first.Process.Pid)

	if err != nil {
		return err
	}

	b, err := cpuTime(second.Process.Pid)

	if err != nil {
		return err
	}

	percent := (200*a/(a+b) + 5) / 10 * 5
	fmt.Printf("nice 0 and nice %d on 1 CPU: %d%% and %d%%\n", nice, percent, 100-percent)

	return nil
}

Listing 6.6 Measuring the division of a CPU, in Rust. code/rust/schedfacts/src/main.rs

/// Ends a busy process and collects it.
fn stop(mut child: Child) -> io::Result<()> {
    child.kill()?;
    child.wait()?;

    Ok(())
}

/// Lets a process of nice 0 and a process of the given nice value compete for
/// CPU 0 for 2 seconds and prints the share of CPU time each received,
/// rounded to the nearest 5%.
fn share(nice: i32) -> io::Result<()> {
    let first = spin(0)?;
    let second = spin(nice)?;

    thread::sleep(Duration::from_secs(2));

    let times = cpu_time(first.id()).and_then(|a| Ok((a, cpu_time(second.id())?)));

    stop(first)?;
    stop(second)?;

    let (a, b) = times?;
    let percent = (200 * a / (a + b) + 5) / 10 * 5;

    println!(
        "nice 0 and nice {nice} on 1 CPU: {percent}% and {}%",
        100 - percent
    );

    Ok(())
}

nice 0 and nice 0 on 1 CPU: 50% and 50%
nice 0 and nice 5 on 1 CPU: 75% and 25%
nice 0 and nice 10 on 1 CPU: 90% and 10%

The shares predicted from the table are 1024/2048 = 50%, 1024/1359 = 75.3% and 1024/1134 = 90.3%. The measurement agrees with all of them.

Virtual runtime

The scheduler needs a rule that produces these shares without computing them. It keeps a number for each task, the virtual runtime or vruntime, and follows 2 rules.

1. When a task has run for a time t, its vruntime increases by t × 1024 / weight.
2. The task with the smallest vruntime runs next.

For a task of weight 1024, vruntime is the real time it has run. A task of weight 335 is charged about 3.06 ms for each millisecond, and a task of weight 3121 is charged about 0.33 ms.

To see why this produces the shares, take a task A of weight 1024 and a task C of weight 335. Rule 2 always runs the one that is behind, so their vruntimes stay close together. Suppose both have advanced by 300 ms of vruntime. A was charged at the real rate, so it ran for 300 ms. C was charged 3.06 ms per millisecond, so it ran for 98 ms. The ratio 300 to 98 is the ratio of the weights, 1024 to 335. Equal progress in virtual time means weighted progress in real time.

The kernel’s structures

The scheduling data of a task is a struct sched_entity, embedded in its task_struct as the field se. Each CPU has a struct cfs_rq, the run queue of the fair class. These are the definitions in Linux 5.15, from include/linux/sched.h and kernel/sched/sched.h, reduced to the fields we use.

struct load_weight {
        unsigned long weight;                  /* 1024 for nice 0 */
        u32 inv_weight;                        /* precomputed 2^32 / weight */
};

struct sched_entity {
        struct load_weight load;
        struct rb_node run_node;               /* its node in the run queue's tree */
        unsigned int on_rq;                    /* 1 while it is in the run queue */
        u64 exec_start;                        /* when it was last put on a CPU */
        u64 sum_exec_runtime;                  /* real CPU time received, in ns */
        u64 vruntime;                          /* virtual runtime, in ns */
        /* ... */
};

struct cfs_rq {
        struct load_weight load;               /* sum of the weights in the queue */
        unsigned int nr_running;               /* number of runnable entities */
        u64 min_vruntime;                      /* a floor that only increases */
        struct rb_root_cached tasks_timeline;  /* the tree, ordered by vruntime */
        struct sched_entity *curr;             /* the entity on the CPU */
        /* ... */
};

Rule 2 needs a data structure that returns its smallest element and accepts insertions, both quickly. A red-black tree does each in time proportional to the logarithm of the number of tasks, and the kernel keeps a pointer to the leftmost node, so finding the next task takes constant time.

The length of a slice

How long a task runs before the scheduler chooses again is also derived from the weights. The scheduler aims to run every runnable task once within a period called the scheduling latency, and gives each task the part of the period that corresponds to its weight. With more tasks the parts become shorter, down to a minimum, so that the cost of switching does not dominate. The defaults in Linux 5.15 are a latency of 6 ms and a minimum of 0.75 ms, both multiplied by a factor that grows with the number of CPUs: 4 on a machine with 8 or more CPUs, giving 24 ms and 3 ms here.

An example with the 3 tasks of this chapter makes the rule concrete. A and B have weight 1024 and C has weight 335, a total of 2383. In a period of 24 ms, A receives 24 × 1024 / 2383 = 10.3 ms, B the same, and C receives 24 × 335 / 2383 = 3.4 ms.

Modelling the scheduler

The weights are the kernel’s table sched_prio_to_weight. The model uses a fixed slice of 1 ms, which is enough to reproduce the shares.

Listing 6.7 The imports of cfs, in Go. code/go/cfs/main.go

import (
	"container/heap"
	"fmt"
	"strings"
)

Listing 6.8 The imports of cfs, in Rust. code/rust/cfs/src/main.rs

use std::collections::BTreeSet;

Listing 6.9 Weights and tasks, in Go. code/go/cfs/main.go

// weights is the kernel's table sched_prio_to_weight: the weight of each nice
// value from -20 to 19. Each step of nice changes the weight by about 25%.
var weights = [40]int64{
	88761, 71755, 56483, 46273, 36291, 29154, 23254, 18705, 14949, 11916,
	9548, 7620, 6100, 4904, 3906, 3121, 2501, 1991, 1586, 1277,
	1024, 820, 655, 526, 423, 335, 272, 215, 172, 137,
	110, 87, 70, 56, 45, 36, 29, 23, 18, 15,
}

// slice is the time a task runs before the scheduler chooses again, in
// nanoseconds.
const slice = 1_000_000

// A task is one schedulable entity. vruntime is its virtual runtime in
// nanoseconds, and slices counts how often it has run.
type task struct {
	name     string
	weight   int64
	vruntime int64
	slices   int
}

func newTask(name string, nice int) *task {
	return &task{name: name, weight: weights[nice+20]}
}

Listing 6.10 Weights and tasks, in Rust. code/rust/cfs/src/main.rs

/// The kernel's table `sched_prio_to_weight`: the weight of each nice value
/// from -20 to 19. Each step of nice changes the weight by about 25%.
const WEIGHTS: [u64; 40] = [
    88761, 71755, 56483, 46273, 36291, 29154, 23254, 18705, 14949, 11916, 9548, 7620, 6100, 4904,
    3906, 3121, 2501, 1991, 1586, 1277, 1024, 820, 655, 526, 423, 335, 272, 215, 172, 137, 110, 87,
    70, 56, 45, 36, 29, 23, 18, 15,
];

/// The time a task runs before the scheduler chooses again, in nanoseconds.
const SLICE: u64 = 1_000_000;

/// One schedulable entity. `vruntime` is its virtual runtime in nanoseconds,
/// and `slices` counts how often it has run.
struct Task {
    name: &'static str,
    weight: u64,
    vruntime: u64,
    slices: u64,
}

impl Task {
    fn new(name: &'static str, nice: i32) -> Task {
        Task {
            name,
            weight: WEIGHTS[(nice + 20) as usize],
            vruntime: 0,
            slices: 0,
        }
    }
}

The run queue orders tasks by vruntime. The Go version is a binary heap from container/heap, and the Rust version is a BTreeSet of keys. Both give the operations rule 2 needs.

Listing 6.11 The run queue, in Go. code/go/cfs/main.go

// A queue holds the runnable tasks ordered by virtual runtime, with the name
// as a tie-break. It implements heap.Interface. The kernel keeps the same
// order in a red-black tree and takes the leftmost node.
type queue []*task

func (q queue) Len() int      { return len(q) }
func (q queue) Swap(i, j int) { q[i], q[j] = q[j], q[i] }
func (q *queue) Push(t any)   { *q = append(*q, t.(*task)) }

func (q queue) Less(i, j int) bool {
	if q[i].vruntime != q[j].vruntime {
		return q[i].vruntime < q[j].vruntime
	}

	return q[i].name < q[j].name
}

func (q *queue) Pop() any {
	old := *q
	last := old[len(old)-1]
	*q = old[:len(old)-1]

	return last
}

Listing 6.12 The run queue, in Rust. code/rust/cfs/src/main.rs

/// The runnable tasks, ordered by virtual runtime with the name as a
/// tie-break. `runnable` holds the order and `tasks` holds the tasks, indexed
/// by the number in the key. The kernel keeps the same order in a red-black
/// tree and takes the leftmost node.
#[derive(Default)]
struct Scheduler {
    tasks: Vec<Task>,
    runnable: BTreeSet<(u64, &'static str, usize)>,
    min_vruntime: u64,
}

impl Scheduler {
    fn enqueue(&mut self, id: usize) {
        let task = &self.tasks[id];

        self.runnable.insert((task.vruntime, task.name, id));
    }
}

tick applies both rules. minVruntime follows the vruntime of the task at the front of the queue.

Listing 6.13 Running the task with the smallest vruntime, in Go. code/go/cfs/main.go

// A scheduler is a run queue and the smallest virtual runtime it has seen,
// which only increases.
type scheduler struct {
	runnable    queue
	minVruntime int64
}

// tick runs the task with the smallest virtual runtime for one slice and
// returns it. The task is charged slice * 1024 / weight, so a task of weight
// 1024 is charged real time and a heavier task is charged less.
func (s *scheduler) tick() *task {
	t := heap.Pop(&s.runnable).(*task)

	t.vruntime += slice * 1024 / t.weight
	t.slices++

	heap.Push(&s.runnable, t)
	s.minVruntime = max(s.minVruntime, s.runnable[0].vruntime)

	return t
}

// wake makes a task runnable. With place set, a task that slept is moved
// forward to the queue's minimum, so that it cannot claim the time it did
// not use while it slept.
func (s *scheduler) wake(t *task, place bool) {
	if place {
		t.vruntime = max(t.vruntime, s.minVruntime)
	}

	heap.Push(&s.runnable, t)
}

Listing 6.14 Running the task with the smallest vruntime, in Rust. code/rust/cfs/src/main.rs

impl Scheduler {
    /// Runs the task with the smallest virtual runtime for one slice and
    /// returns its number. The task is charged `SLICE * 1024 / weight`, so a
    /// task of weight 1024 is charged real time and a heavier task is charged
    /// less.
    fn tick(&mut self) -> Option<usize> {
        let (_, _, id) = self.runnable.pop_first()?;
        let task = &mut self.tasks[id];

        task.vruntime += SLICE * 1024 / task.weight;
        task.slices += 1;

        self.enqueue(id);

        if let Some(&(leftmost, _, _)) = self.runnable.first() {
            self.min_vruntime = self.min_vruntime.max(leftmost);
        }

        Some(id)
    }

    /// Makes a task runnable and returns its number. With `place` set, a task
    /// that slept is moved forward to the queue's minimum, so that it cannot
    /// claim the time it did not use while it slept.
    fn wake(&mut self, mut task: Task, place: bool) -> usize {
        if place {
            task.vruntime = task.vruntime.max(self.min_vruntime);
        }

        self.tasks.push(task);
        self.enqueue(self.tasks.len() - 1);

        self.tasks.len() - 1
    }
}

share repeats the measurement on the model: a task of nice 0 against a task of another nice value, for 2000 slices.

Listing 6.15 The shares on the model, in Go. code/go/cfs/main.go

// share runs a task of nice 0 against a task of the given nice value for
// 2000 slices and prints each task's share, rounded to the nearest 5%.
func share(nice int) {
	s := &scheduler{}
	first, second := newTask("A", 0), newTask("B", nice)

	s.wake(first, true)
	s.wake(second, true)

	for range 2000 {
		s.tick()
	}

	percent := (200*first.slices/(first.slices+second.slices) + 5) / 10 * 5
	fmt.Printf("nice 0 and nice %d on 1 CPU: %d%% and %d%%\n", nice, percent, 100-percent)
}

Listing 6.16 The shares on the model, in Rust. code/rust/cfs/src/main.rs

/// Runs a task of nice 0 against a task of the given nice value for 2000
/// slices and prints each task's share, rounded to the nearest 5%.
fn share(nice: i32) {
    let mut scheduler = Scheduler::default();

    let first = scheduler.wake(Task::new("A", 0), true);
    let second = scheduler.wake(Task::new("B", nice), true);

    for _ in 0..2000 {
        scheduler.tick();
    }

    let (a, b) = (
        scheduler.tasks[first].slices,
        scheduler.tasks[second].slices,
    );
    let percent = (200 * a / (a + b) + 5) / 10 * 5;

    println!(
        "nice 0 and nice {nice} on 1 CPU: {percent}% and {}%",
        100 - percent
    );
}

order prints which task runs in each of the first 12 slices when A and B have nice 0 and C has nice 5.

Listing 6.17 The order of the first slices, in Go. code/go/cfs/main.go

// order prints which task runs in each of the first 12 slices.
func order() {
	s := &scheduler{}

	for _, t := range []*task{newTask("A", 0), newTask("B", 0), newTask("C", 5)} {
		s.wake(t, true)
	}

	var names []string

	for range 12 {
		names = append(names, s.tick().name)
	}

	fmt.Printf("A and B at nice 0, C at nice 5: %s\n", strings.Join(names, " "))
}

Listing 6.18 The order of the first slices, in Rust. code/rust/cfs/src/main.rs

/// Prints which task runs in each of the first 12 slices.
fn order() {
    let mut scheduler = Scheduler::default();

    for task in [Task::new("A", 0), Task::new("B", 0), Task::new("C", 5)] {
        scheduler.wake(task, true);
    }

    let mut names = Vec::new();

    for _ in 0..12 {
        if let Some(id) = scheduler.tick() {
            names.push(scheduler.tasks[id].name);
        }
    }

    println!("A and B at nice 0, C at nice 5: {}", names.join(" "));
}

A task that wakes up

A task that sleeps is removed from the run queue, and its vruntime stops increasing. After a long sleep its vruntime is far below that of the tasks that kept running. Under rule 2 alone it would then run without interruption until it had caught up, and every other task would stop for that time. A task that had slept for an hour could claim the CPU for many minutes.

The scheduler prevents this when the task wakes: it raises the task’s vruntime to the min_vruntime of the queue. The task is treated as if it had kept pace while it slept. It competes on equal terms from the moment it wakes and receives no credit for the time it did not use.

Listing 6.19 Waking a task with and without placement, in Go. code/go/cfs/main.go

// sleeper lets A and B run for 200 slices, wakes C with a virtual runtime of
// 0, and counts how many of the next 30 slices C receives.
func sleeper(place bool) int {
	s := &scheduler{}

	s.wake(newTask("A", 0), true)
	s.wake(newTask("B", 0), true)

	for range 200 {
		s.tick()
	}

	c := newTask("C", 0)
	s.wake(c, place)

	for range 30 {
		s.tick()
	}

	return c.slices
}

Listing 6.20 Waking a task with and without placement, in Rust. code/rust/cfs/src/main.rs

/// Lets A and B run for 200 slices, wakes C with a virtual runtime of 0, and
/// counts how many of the next 30 slices C receives.
fn sleeper(place: bool) -> u64 {
    let mut scheduler = Scheduler::default();

    scheduler.wake(Task::new("A", 0), true);
    scheduler.wake(Task::new("B", 0), true);

    for _ in 0..200 {
        scheduler.tick();
    }

    let c = scheduler.wake(Task::new("C", 0), place);

    for _ in 0..30 {
        scheduler.tick();
    }

    scheduler.tasks[c].slices
}

nice 0 and nice 0 on 1 CPU: 50% and 50%
nice 0 and nice 5 on 1 CPU: 75% and 25%
nice 0 and nice 10 on 1 CPU: 90% and 10%
A and B at nice 0, C at nice 5: A B C A B A B A B C A B
C wakes with its old virtual runtime: it receives 30 of the next 30 slices
C wakes and is placed at the minimum: it receives 10 of the next 30 slices

Several CPUs

Everything so far concerned a single CPU. A machine with several CPUs has a run queue for each, with its own tree and its own lock, so that the CPUs can schedule without waiting for each other. A task is in the queue of exactly 1 CPU at a time.

Separate queues can become uneven, with tasks waiting on one CPU while another is idle. Load balancing corrects this: periodically, and whenever a CPU runs out of tasks, the kernel compares the queues and moves tasks from a loaded CPU to a lighter one. Moving has a cost, because the task’s data is in the caches of the CPU it leaves, so the balancer prefers to leave a task where it last ran.

A program can restrict the CPUs a task may use, its affinity, with sched_setaffinity or the taskset command. schedfacts used this to force its 2 processes onto the same CPU.

$ taskset -p 577022
pid 577022's current affinity mask: ff

The mask ff has 8 bits set: the task may run on any of the 8 CPUs.

Inspecting the scheduler on a live system

A high count of involuntary switches means the task wants more CPU than it gets. A high count of voluntary switches means it mostly waits.

The complete programs

Listing 6.21 The complete schedfacts, in Go. code/go/schedfacts/main.go

// Command schedfacts measures how the scheduler divides a CPU between busy
// processes of different nice values. It starts the processes on the same
// CPU, lets them compete, and reads the CPU time each one received.
//
// Run it from code/go with:
//
//	go run ./schedfacts
package main

import (
	"fmt"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"time"
)


// spin starts a busy shell loop, restricted to CPU 0 and with
// the given nice value.
func spin(nice int) (*exec.Cmd, error) {
	child := exec.Command("taskset", "-c", "0", "nice", "-n", strconv.Itoa(nice), "sh", "-c", "while :; do :; done")

	return child, child.Start()
}

// cpuTime returns the CPU time a process has used, in clock ticks: fields 14
// and 15 of /proc/<pid>/stat, the time in user mode and in kernel mode.
func cpuTime(pid int) (int, error) {
	stat, err := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))

	if err != nil {
		return 0, err
	}

	fields := strings.Fields(string(stat[strings.LastIndexByte(string(stat), ')')+1:]))

	if len(fields) < 13 {
		return 0, fmt.Errorf("/proc/%d/stat: too few fields", pid)
	}

	var user, kernel int

	if _, err := fmt.Sscan(fields[11], &user); err != nil {
		return 0, err
	}

	if _, err := fmt.Sscan(fields[12], &kernel); err != nil {
		return 0, err
	}

	return user + kernel, nil
}


// stop ends a busy process and collects it.
func stop(child *exec.Cmd) {
	child.Process.Kill()
	child.Wait()
}

// share lets a process of nice 0 and a process of the given nice value
// compete for CPU 0 for 2 seconds and prints the share of CPU time each
// received, rounded to the nearest 5%.
func share(nice int) error {
	first, err := spin(0)

	if err != nil {
		return err
	}

	defer stop(first)

	second, err := spin(nice)

	if err != nil {
		return err
	}

	defer stop(second)

	time.Sleep(2 * time.Second)

	a, err := cpuTime(first.Process.Pid)

	if err != nil {
		return err
	}

	b, err := cpuTime(second.Process.Pid)

	if err != nil {
		return err
	}

	percent := (200*a/(a+b) + 5) / 10 * 5
	fmt.Printf("nice 0 and nice %d on 1 CPU: %d%% and %d%%\n", nice, percent, 100-percent)

	return nil
}


func main() {
	for _, nice := range []int{0, 5, 10} {
		if err := share(nice); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 6.22 The complete schedfacts, in Rust. code/rust/schedfacts/src/main.rs

//! Measures how the scheduler divides a CPU between busy processes of
//! different nice values. It starts the processes on the same CPU, lets them
//! compete, and reads the CPU time each one received.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p schedfacts
//! ```

use std::fs;
use std::io;
use std::process::{Child, Command};
use std::thread;
use std::time::Duration;

/// Starts a busy shell loop, restricted to CPU 0 and with the
/// given nice value.
fn spin(nice: i32) -> io::Result<Child> {
    Command::new("taskset")
        .args(["-c", "0", "nice", "-n", &nice.to_string()])
        .args(["sh", "-c", "while :; do :; done"])
        .spawn()
}

/// Returns the CPU time a process has used, in clock ticks: fields 14 and 15
/// of `/proc/<pid>/stat`, the time in user mode and in kernel mode.
fn cpu_time(pid: u32) -> io::Result<u64> {
    let stat = fs::read_to_string(format!("/proc/{pid}/stat"))?;
    let rest = stat.rsplit_once(')').map_or("", |(_, rest)| rest);

    let ticks: Vec<u64> = rest
        .split_whitespace()
        .skip(11)
        .take(2)
        .filter_map(|field| field.parse().ok())
        .collect();

    match ticks.as_slice() {
        [user, kernel] => Ok(user + kernel),
        _ => Err(io::ErrorKind::InvalidData.into()),
    }
}

/// Ends a busy process and collects it.
fn stop(mut child: Child) -> io::Result<()> {
    child.kill()?;
    child.wait()?;

    Ok(())
}

/// Lets a process of nice 0 and a process of the given nice value compete for
/// CPU 0 for 2 seconds and prints the share of CPU time each received,
/// rounded to the nearest 5%.
fn share(nice: i32) -> io::Result<()> {
    let first = spin(0)?;
    let second = spin(nice)?;

    thread::sleep(Duration::from_secs(2));

    let times = cpu_time(first.id()).and_then(|a| Ok((a, cpu_time(second.id())?)));

    stop(first)?;
    stop(second)?;

    let (a, b) = times?;
    let percent = (200 * a / (a + b) + 5) / 10 * 5;

    println!(
        "nice 0 and nice {nice} on 1 CPU: {percent}% and {}%",
        100 - percent
    );

    Ok(())
}

fn main() -> io::Result<()> {
    for nice in [0, 5, 10] {
        share(nice)?;
    }

    Ok(())
}

Listing 6.23 The complete cfs, in Go. code/go/cfs/main.go

// Command cfs models the Completely Fair Scheduler: every runnable task has a
// virtual runtime, the scheduler always runs the task with the smallest one,
// and a task's virtual runtime advances more slowly the larger its weight.
//
// Run it from code/go with:
//
//	go run ./cfs
package main

import (
	"container/heap"
	"fmt"
	"strings"
)


// weights is the kernel's table sched_prio_to_weight: the weight of each nice
// value from -20 to 19. Each step of nice changes the weight by about 25%.
var weights = [40]int64{
	88761, 71755, 56483, 46273, 36291, 29154, 23254, 18705, 14949, 11916,
	9548, 7620, 6100, 4904, 3906, 3121, 2501, 1991, 1586, 1277,
	1024, 820, 655, 526, 423, 335, 272, 215, 172, 137,
	110, 87, 70, 56, 45, 36, 29, 23, 18, 15,
}

// slice is the time a task runs before the scheduler chooses again, in
// nanoseconds.
const slice = 1_000_000

// A task is one schedulable entity. vruntime is its virtual runtime in
// nanoseconds, and slices counts how often it has run.
type task struct {
	name     string
	weight   int64
	vruntime int64
	slices   int
}

func newTask(name string, nice int) *task {
	return &task{name: name, weight: weights[nice+20]}
}


// A queue holds the runnable tasks ordered by virtual runtime, with the name
// as a tie-break. It implements heap.Interface. The kernel keeps the same
// order in a red-black tree and takes the leftmost node.
type queue []*task

func (q queue) Len() int      { return len(q) }
func (q queue) Swap(i, j int) { q[i], q[j] = q[j], q[i] }
func (q *queue) Push(t any)   { *q = append(*q, t.(*task)) }

func (q queue) Less(i, j int) bool {
	if q[i].vruntime != q[j].vruntime {
		return q[i].vruntime < q[j].vruntime
	}

	return q[i].name < q[j].name
}

func (q *queue) Pop() any {
	old := *q
	last := old[len(old)-1]
	*q = old[:len(old)-1]

	return last
}


// A scheduler is a run queue and the smallest virtual runtime it has seen,
// which only increases.
type scheduler struct {
	runnable    queue
	minVruntime int64
}

// tick runs the task with the smallest virtual runtime for one slice and
// returns it. The task is charged slice * 1024 / weight, so a task of weight
// 1024 is charged real time and a heavier task is charged less.
func (s *scheduler) tick() *task {
	t := heap.Pop(&s.runnable).(*task)

	t.vruntime += slice * 1024 / t.weight
	t.slices++

	heap.Push(&s.runnable, t)
	s.minVruntime = max(s.minVruntime, s.runnable[0].vruntime)

	return t
}

// wake makes a task runnable. With place set, a task that slept is moved
// forward to the queue's minimum, so that it cannot claim the time it did
// not use while it slept.
func (s *scheduler) wake(t *task, place bool) {
	if place {
		t.vruntime = max(t.vruntime, s.minVruntime)
	}

	heap.Push(&s.runnable, t)
}


// share runs a task of nice 0 against a task of the given nice value for
// 2000 slices and prints each task's share, rounded to the nearest 5%.
func share(nice int) {
	s := &scheduler{}
	first, second := newTask("A", 0), newTask("B", nice)

	s.wake(first, true)
	s.wake(second, true)

	for range 2000 {
		s.tick()
	}

	percent := (200*first.slices/(first.slices+second.slices) + 5) / 10 * 5
	fmt.Printf("nice 0 and nice %d on 1 CPU: %d%% and %d%%\n", nice, percent, 100-percent)
}


// order prints which task runs in each of the first 12 slices.
func order() {
	s := &scheduler{}

	for _, t := range []*task{newTask("A", 0), newTask("B", 0), newTask("C", 5)} {
		s.wake(t, true)
	}

	var names []string

	for range 12 {
		names = append(names, s.tick().name)
	}

	fmt.Printf("A and B at nice 0, C at nice 5: %s\n", strings.Join(names, " "))
}


// sleeper lets A and B run for 200 slices, wakes C with a virtual runtime of
// 0, and counts how many of the next 30 slices C receives.
func sleeper(place bool) int {
	s := &scheduler{}

	s.wake(newTask("A", 0), true)
	s.wake(newTask("B", 0), true)

	for range 200 {
		s.tick()
	}

	c := newTask("C", 0)
	s.wake(c, place)

	for range 30 {
		s.tick()
	}

	return c.slices
}


func main() {
	for _, nice := range []int{0, 5, 10} {
		share(nice)
	}

	order()

	fmt.Printf("C wakes with its old virtual runtime: it receives %d of the next 30 slices\n", sleeper(false))
	fmt.Printf("C wakes and is placed at the minimum: it receives %d of the next 30 slices\n", sleeper(true))
}

Listing 6.24 The complete cfs, in Rust. code/rust/cfs/src/main.rs

//! Models the Completely Fair Scheduler: every runnable task has a virtual
//! runtime, the scheduler always runs the task with the smallest one, and a
//! task's virtual runtime advances more slowly the larger its weight.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p cfs
//! ```

use std::collections::BTreeSet;

/// The kernel's table `sched_prio_to_weight`: the weight of each nice value
/// from -20 to 19. Each step of nice changes the weight by about 25%.
const WEIGHTS: [u64; 40] = [
    88761, 71755, 56483, 46273, 36291, 29154, 23254, 18705, 14949, 11916, 9548, 7620, 6100, 4904,
    3906, 3121, 2501, 1991, 1586, 1277, 1024, 820, 655, 526, 423, 335, 272, 215, 172, 137, 110, 87,
    70, 56, 45, 36, 29, 23, 18, 15,
];

/// The time a task runs before the scheduler chooses again, in nanoseconds.
const SLICE: u64 = 1_000_000;

/// One schedulable entity. `vruntime` is its virtual runtime in nanoseconds,
/// and `slices` counts how often it has run.
struct Task {
    name: &'static str,
    weight: u64,
    vruntime: u64,
    slices: u64,
}

impl Task {
    fn new(name: &'static str, nice: i32) -> Task {
        Task {
            name,
            weight: WEIGHTS[(nice + 20) as usize],
            vruntime: 0,
            slices: 0,
        }
    }
}

/// The runnable tasks, ordered by virtual runtime with the name as a
/// tie-break. `runnable` holds the order and `tasks` holds the tasks, indexed
/// by the number in the key. The kernel keeps the same order in a red-black
/// tree and takes the leftmost node.
#[derive(Default)]
struct Scheduler {
    tasks: Vec<Task>,
    runnable: BTreeSet<(u64, &'static str, usize)>,
    min_vruntime: u64,
}

impl Scheduler {
    fn enqueue(&mut self, id: usize) {
        let task = &self.tasks[id];

        self.runnable.insert((task.vruntime, task.name, id));
    }
}

impl Scheduler {
    /// Runs the task with the smallest virtual runtime for one slice and
    /// returns its number. The task is charged `SLICE * 1024 / weight`, so a
    /// task of weight 1024 is charged real time and a heavier task is charged
    /// less.
    fn tick(&mut self) -> Option<usize> {
        let (_, _, id) = self.runnable.pop_first()?;
        let task = &mut self.tasks[id];

        task.vruntime += SLICE * 1024 / task.weight;
        task.slices += 1;

        self.enqueue(id);

        if let Some(&(leftmost, _, _)) = self.runnable.first() {
            self.min_vruntime = self.min_vruntime.max(leftmost);
        }

        Some(id)
    }

    /// Makes a task runnable and returns its number. With `place` set, a task
    /// that slept is moved forward to the queue's minimum, so that it cannot
    /// claim the time it did not use while it slept.
    fn wake(&mut self, mut task: Task, place: bool) -> usize {
        if place {
            task.vruntime = task.vruntime.max(self.min_vruntime);
        }

        self.tasks.push(task);
        self.enqueue(self.tasks.len() - 1);

        self.tasks.len() - 1
    }
}

/// Runs a task of nice 0 against a task of the given nice value for 2000
/// slices and prints each task's share, rounded to the nearest 5%.
fn share(nice: i32) {
    let mut scheduler = Scheduler::default();

    let first = scheduler.wake(Task::new("A", 0), true);
    let second = scheduler.wake(Task::new("B", nice), true);

    for _ in 0..2000 {
        scheduler.tick();
    }

    let (a, b) = (
        scheduler.tasks[first].slices,
        scheduler.tasks[second].slices,
    );
    let percent = (200 * a / (a + b) + 5) / 10 * 5;

    println!(
        "nice 0 and nice {nice} on 1 CPU: {percent}% and {}%",
        100 - percent
    );
}

/// Prints which task runs in each of the first 12 slices.
fn order() {
    let mut scheduler = Scheduler::default();

    for task in [Task::new("A", 0), Task::new("B", 0), Task::new("C", 5)] {
        scheduler.wake(task, true);
    }

    let mut names = Vec::new();

    for _ in 0..12 {
        if let Some(id) = scheduler.tick() {
            names.push(scheduler.tasks[id].name);
        }
    }

    println!("A and B at nice 0, C at nice 5: {}", names.join(" "));
}

/// Lets A and B run for 200 slices, wakes C with a virtual runtime of 0, and
/// counts how many of the next 30 slices C receives.
fn sleeper(place: bool) -> u64 {
    let mut scheduler = Scheduler::default();

    scheduler.wake(Task::new("A", 0), true);
    scheduler.wake(Task::new("B", 0), true);

    for _ in 0..200 {
        scheduler.tick();
    }

    let c = scheduler.wake(Task::new("C", 0), place);

    for _ in 0..30 {
        scheduler.tick();
    }

    scheduler.tasks[c].slices
}

fn main() {
    for nice in [0, 5, 10] {
        share(nice);
    }

    order();

    println!(
        "C wakes with its old virtual runtime: it receives {} of the next 30 slices",
        sleeper(false)
    );
    println!(
        "C wakes and is placed at the minimum: it receives {} of the next 30 slices",
        sleeper(true)
    );
}

References

• sched(7), nice(2), sched_setaffinity(2) and chrt(1), Linux manual pages: https://man7.org/linux/man-pages/man7/sched.7.html
• CFS Scheduler, Linux kernel documentation: https://www.kernel.org/doc/html/v5.15/scheduler/sched-design-CFS.html
• kernel/sched/fair.c, kernel/sched/core.c and include/linux/sched.h, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/kernel/sched/fair.c

Virtual Memory

The address space of a process is the set of memory addresses the process can use and the meaning the kernel has given to each. The addresses are virtual: they are translated to locations in physical memory by the processor, through a table that the kernel maintains for each process. Different processes can use the same virtual address for different memory, and neither can reach the memory of the other.

In this chapter, we will learn how the kernel fills an address space. We will first measure with vmfacts when a mapped page receives physical memory and where a write through a file mapping ends up. Then we will write vmmodel, a model with mappings, a page table and a page fault handler, and use it to follow copy-on-write after fork.

Why addresses are virtual

Suppose programs used the addresses of physical memory directly. A program compiled to keep a variable at address 0x1000 could run only when that memory was free, and 2 copies of it could not run together. Any program could read or overwrite the memory of any other by using its addresses. And a program could use no more memory than was physically free at the moment it started.

Virtual addresses remove all 3 problems with a single idea: the addresses a program uses are private to it, and a table per process says which physical memory, if any, stands behind each of them.

With this arrangement every program can be built as if it had the machine to itself. The kernel decides which physical memory each address reaches, and it can decide late, at the moment the address is first used. The rest of the chapter is about how the table is organised and when it is filled in.

The layout of an address space

Before the mechanism, we look at what an address space contains. The kernel lists the address ranges a process can use in /proc/<pid>/maps. This is the listing of a cat process on our machine, with 3 of its columns removed.

$ cat /proc/self/maps
aaaab7c80000-aaaab7c86000 r-xp 00000000 /usr/bin/cat
aaaab7c95000-aaaab7c96000 r--p 00005000 /usr/bin/cat
aaaab7c96000-aaaab7c97000 rw-p 00006000 /usr/bin/cat
aaaacae31000-aaaacae52000 rw-p 00000000 [heap]
ffffaa650000-ffffaa7d8000 r-xp 00000000 /usr/lib/aarch64-linux-gnu/libc.so.6
ffffaa7e7000-ffffaa7eb000 r--p 00187000 /usr/lib/aarch64-linux-gnu/libc.so.6
ffffaa7eb000-ffffaa7ed000 rw-p 0018b000 /usr/lib/aarch64-linux-gnu/libc.so.6
ffffaa80b000-ffffaa82d000 rw-p 00000000
ffffaa82d000-ffffaa858000 r-xp 00000000 /usr/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
ffffaa866000-ffffaa867000 r-xp 00000000 [vdso]
fffff2ebe000-fffff2ee0000 rw-p 00000000 [stack]

The numbers are addresses in hexadecimal, and they are less forbidding than they look. A line gives a start and an end, and the size of the range is the difference. For the first line, 0xaaaab7c86000 - 0xaaaab7c80000 = 0x6000, which is 6 × 4096 bytes: the code of cat occupies 6 pages. Only the differences carry information. The absolute values change on every run.

Each line is a mapping: a range of addresses with the same permissions and the same source.

The ranges are far apart, and most of the space between them is unmapped. An access to an unmapped address ends the process with SIGSEGV. The addresses differ from run to run, because the kernel chooses the positions of the regions at random for each new program, which makes attacks that depend on known addresses harder.

Pages, mappings and the page table

Virtual and physical memory are both divided into units of the same size, 4096 bytes on this machine. A unit of virtual memory is a page, and a unit of physical memory is a frame. The address space is described at 2 levels.

Translating an address

Translation is easiest to follow on a machine small enough to work by hand. Imagine addresses of 16 bits and pages of 256 bytes. An address then has 2 hexadecimal digits for the page number and 2 for the offset within the page. To translate 0x1234, split it into page 0x12 and offset 0x34, look up entry 0x12 of the page table, find frame 0x7A there, and attach the offset: the physical address is 0x7A34.

A real machine does exactly this, with larger numbers and with a table that is split into several levels to save space.

The processor translates every address a program uses, in hardware. An address of this machine has 48 significant bits. The low 12 bits are the offset within a page, since a page has 4096 = 2¹² bytes. The remaining 36 bits are the page number, and they are used in 4 groups of 9 bits.

The page table is a tree of tables, each a page of 512 entries. The processor starts at the root table of the process, uses the first 9 bits to select an entry, follows it to a table of the next level, and repeats this 4 times. The last entry holds the number of the frame and the permission bits. The frame number and the offset together are the physical address.

A tree is used because the address space is almost empty. A flat table for 2³⁶ pages would need 512 GiB for each process. In the tree, a table exists only for the regions that are in use, and the cat process above needs a few dozen kilobytes.

Walking 4 tables for every memory access would be slow, so the processor keeps the results of recent translations in a cache, the translation lookaside buffer or TLB. Most accesses are answered from the TLB. When the kernel changes a page table entry, it tells the processor to discard the cached translation.

mmap adds a mapping. It writes no page table entry and allocates no frame. A mapping is a promise: these addresses are valid, and this is what they contain.

The promise is kept later. When the program uses an address whose page has no entry, or writes where the entry forbids writing, the processor cannot complete the instruction. It raises a page fault, which transfers control to the kernel. The kernel looks up the address in the mappings. If a mapping permits the access, the kernel provides a frame, writes the entry, and returns, and the processor repeats the instruction. If no mapping permits it, the kernel sends the process the signal SIGSEGV.

Filling the table on demand means that memory is spent only on pages that are used. A program can map a large range and touch a small part of it, and a large program starts quickly because its code is read from disk page by page as it runs.

The kernel’s structures

The kernel describes an address space with 3 structures. These are the definitions in include/linux/mm_types.h of Linux 5.15, reduced to the fields we use.

struct mm_struct {
        struct vm_area_struct *mmap;           /* the mappings, as a list */
        struct rb_root mm_rb;                  /* the mappings, as a tree by address */
        pgd_t *pgd;                            /* the root of the page table */
        int map_count;                         /* number of mappings */
        unsigned long total_vm;                /* pages mapped */
        unsigned long start_brk, brk;          /* the heap */
        unsigned long start_stack;             /* the stack */
        /* ... */
};

struct vm_area_struct {
        unsigned long vm_start;                /* first address of the mapping */
        unsigned long vm_end;                  /* first address after it */
        struct vm_area_struct *vm_next, *vm_prev;
        struct mm_struct *vm_mm;               /* the address space it belongs to */
        unsigned long vm_flags;                /* VM_READ, VM_WRITE, VM_EXEC, VM_SHARED */
        unsigned long vm_pgoff;                /* offset in the file, in pages */
        struct file *vm_file;                  /* the mapped file, or NULL */
        /* ... */
};

struct page {
        unsigned long flags;                   /* dirty, locked, ... */
        struct address_space *mapping;         /* the file this frame caches, if any */
        pgoff_t index;                         /* its page number in that file */
        atomic_t _mapcount;                    /* page table entries that map it */
        atomic_t _refcount;                    /* users of the frame */
        /* ... */
};

There are 2 descriptions of the same address space, and the difference between them is the subject of this chapter. The list of VMAs says what the process is allowed to use. The page table says what has actually been given a frame. mmap changes only the first.

Measuring an address space

vmfacts uses golang.org/x/sys/unix in Go and the memmap2 crate in Rust.

$ cargo new vmfacts
$ cargo add -p vmfacts memmap2

The kernel publishes the page table of a process in /proc/self/pagemap: 8 bytes for each virtual page, in which bit 63 is set when the page has a frame. resident counts those pages for a mapping.

Listing 7.1 The imports of vmfacts, in Go. code/go/vmfacts/main.go

import (
	"bufio"
	"encoding/binary"
	"fmt"
	"log"
	"os"
	"path/filepath"
	"reflect"
	"strings"

	"golang.org/x/sys/unix"
)

Listing 7.2 The imports of vmfacts, in Rust. code/rust/vmfacts/src/main.rs

use std::fs::{self, File, OpenOptions};
use std::io;
use std::os::unix::fs::FileExt;
use std::path::Path;

use memmap2::{MmapMut, MmapOptions};

Listing 7.3 Counting the pages of a mapping that have a frame, in Go. code/go/vmfacts/main.go

const pageSize = 4096

// address returns the virtual address of the first byte of a mapping.
func address(mapping []byte) uintptr {
	return reflect.ValueOf(mapping).Pointer()
}

// resident counts the pages of a mapping that have physical memory. The
// kernel publishes 8 bytes per virtual page in /proc/self/pagemap, and bit 63
// of an entry is set when the page is present in memory.
func resident(mapping []byte) (int, error) {
	pagemap, err := os.Open("/proc/self/pagemap")

	if err != nil {
		return 0, err
	}

	defer pagemap.Close()

	entries := make([]byte, 8*len(mapping)/pageSize)

	if _, err := pagemap.ReadAt(entries, int64(address(mapping)/pageSize*8)); err != nil {
		return 0, err
	}

	count := 0

	for i := 0; i < len(entries); i += 8 {
		if binary.LittleEndian.Uint64(entries[i:])>>63 == 1 {
			count++
		}
	}

	return count, nil
}

Listing 7.4 Counting the pages of a mapping that have a frame, in Rust. code/rust/vmfacts/src/main.rs

const PAGE_SIZE: usize = 4096;

/// Counts the pages of a mapping that have physical memory. The kernel
/// publishes 8 bytes per virtual page in `/proc/self/pagemap`, and bit 63 of
/// an entry is set when the page is present in memory.
fn resident(mapping: &[u8]) -> io::Result<usize> {
    let pagemap = File::open("/proc/self/pagemap")?;
    let mut entries = vec![0u8; 8 * mapping.len() / PAGE_SIZE];
    let offset = mapping.as_ptr() as usize / PAGE_SIZE * 8;

    pagemap.read_exact_at(&mut entries, offset as u64)?;

    let (entries, _) = entries.as_chunks::<8>();
    let count = entries.iter().filter(|entry| entry[7] >> 7 == 1).count();

    Ok(count)
}

demand maps 64 pages of anonymous memory and counts the resident pages 3 times: after mmap, after writing to 10 pages, and after reading 1 further page.

Listing 7.5 When a page receives memory, in Go. code/go/vmfacts/main.go

// demand maps 64 pages of anonymous memory, writes to 10 of them, reads 1
// more, and counts the resident pages after each step.
func demand() error {
	mapping, err := unix.Mmap(-1, 0, 64*pageSize, unix.PROT_READ|unix.PROT_WRITE, unix.MAP_PRIVATE|unix.MAP_ANONYMOUS)

	if err != nil {
		return err
	}

	defer unix.Munmap(mapping)

	before, err := resident(mapping)

	if err != nil {
		return err
	}

	for page := range 10 {
		mapping[page*pageSize] = 1
	}

	written, err := resident(mapping)

	if err != nil {
		return err
	}

	value := mapping[20*pageSize]
	read, err := resident(mapping)

	if err != nil {
		return err
	}

	fmt.Printf("64 pages mapped: %d resident\n", before)
	fmt.Printf("after writing to 10 pages: %d resident\n", written)
	fmt.Printf("after reading 1 more page: %d resident, the byte read is %d\n", read, value)

	return nil
}

Listing 7.6 When a page receives memory, in Rust. code/rust/vmfacts/src/main.rs

/// Maps 64 pages of anonymous memory, writes to 10 of them, reads 1 more, and
/// counts the resident pages after each step.
fn demand() -> io::Result<()> {
    let mut mapping = MmapMut::map_anon(64 * PAGE_SIZE)?;

    let before = resident(&mapping)?;

    for page in 0..10 {
        mapping[page * PAGE_SIZE] = 1;
    }

    let written = resident(&mapping)?;

    let value = mapping[20 * PAGE_SIZE];
    let read = resident(&mapping)?;

    println!("64 pages mapped: {before} resident");
    println!("after writing to 10 pages: {written} resident");
    println!("after reading 1 more page: {read} resident, the byte read is {value}");

    Ok(())
}

protect reads the permissions of a mapping from /proc/self/maps before and after mprotect removes the permission to write.

Listing 7.7 The permissions of a mapping, in Go. code/go/vmfacts/main.go

// permissions returns the permission column of the line in /proc/self/maps
// that starts at the address of a mapping.
func permissions(mapping []byte) (string, error) {
	maps, err := os.Open("/proc/self/maps")

	if err != nil {
		return "", err
	}

	defer maps.Close()

	start := fmt.Sprintf("%x-", address(mapping))

	for lines := bufio.NewScanner(maps); lines.Scan(); {
		if fields := strings.Fields(lines.Text()); strings.HasPrefix(fields[0], start) {
			return fields[1], nil
		}
	}

	return "", fmt.Errorf("mapping at %s not found", start)
}

// protect reads the permissions of a mapping before and after mprotect
// removes the permission to write.
func protect() error {
	mapping, err := unix.Mmap(-1, 0, pageSize, unix.PROT_READ|unix.PROT_WRITE, unix.MAP_PRIVATE|unix.MAP_ANONYMOUS)

	if err != nil {
		return err
	}

	defer unix.Munmap(mapping)

	before, err := permissions(mapping)

	if err != nil {
		return err
	}

	if err := unix.Mprotect(mapping, unix.PROT_READ); err != nil {
		return err
	}

	after, err := permissions(mapping)

	if err != nil {
		return err
	}

	fmt.Printf("permissions of the mapping: %s, after mprotect: %s\n", before, after)

	return nil
}

Listing 7.8 The permissions of a mapping, in Rust. code/rust/vmfacts/src/main.rs

/// Returns the permission column of the line in `/proc/self/maps` that starts
/// at the address of a mapping.
fn permissions(mapping: &[u8]) -> io::Result<String> {
    let start = format!("{:x}-", mapping.as_ptr() as usize);

    fs::read_to_string("/proc/self/maps")?
        .lines()
        .filter(|line| line.starts_with(&start))
        .find_map(|line| line.split_whitespace().nth(1))
        .map(str::to_owned)
        .ok_or_else(|| io::ErrorKind::NotFound.into())
}

/// Reads the permissions of a mapping before and after mprotect removes the
/// permission to write.
fn protect() -> io::Result<()> {
    let mapping = MmapMut::map_anon(PAGE_SIZE)?;
    let before = permissions(&mapping)?;

    let mapping = mapping.make_read_only()?;
    let after = permissions(&mapping)?;

    println!("permissions of the mapping: {before}, after mprotect: {after}");

    Ok(())
}

throughMapping maps a file that contains hello, writes HELLO into the mapped memory, and reads the file again. It does this with a private mapping and with a shared one. Mapping a file is an unsafe operation in Rust, because another process could change the file while the program holds a slice of it. The comment in the listing states why the use is sound here.

Listing 7.9 Writing through a private and a shared file mapping, in Go. code/go/vmfacts/main.go

// throughMapping maps a file that holds "hello", writes "HELLO" through the
// mapping, unmaps it, and returns the contents of the file.
func throughMapping(path string, flags int) (string, error) {
	if err := os.WriteFile(path, []byte("hello"), 0o644); err != nil {
		return "", err
	}

	file, err := os.OpenFile(path, os.O_RDWR, 0)

	if err != nil {
		return "", err
	}

	defer file.Close()

	mapping, err := unix.Mmap(int(file.Fd()), 0, 5, unix.PROT_READ|unix.PROT_WRITE, flags)

	if err != nil {
		return "", err
	}

	copy(mapping, "HELLO")

	if err := unix.Munmap(mapping); err != nil {
		return "", err
	}

	text, err := os.ReadFile(path)

	return string(text), err
}

// files compares a private file mapping with a shared one.
func files() error {
	dir, err := os.MkdirTemp("", "vmfacts")

	if err != nil {
		return err
	}

	defer os.RemoveAll(dir)

	path := filepath.Join(dir, "file")

	for _, kind := range []struct {
		name  string
		flags int
	}{{"private", unix.MAP_PRIVATE}, {"shared", unix.MAP_SHARED}} {
		text, err := throughMapping(path, kind.flags)

		if err != nil {
			return err
		}

		fmt.Printf("HELLO written through a %s file mapping: the file holds %q\n", kind.name, text)
	}

	return nil
}

Listing 7.10 Writing through a private and a shared file mapping, in Rust. code/rust/vmfacts/src/main.rs

/// Maps a file that holds "hello", writes "HELLO" through the mapping, unmaps
/// it, and returns the contents of the file.
fn through_mapping(path: &Path, shared: bool) -> io::Result<String> {
    fs::write(path, "hello")?;

    let file = OpenOptions::new().read(true).write(true).open(path)?;
    let options = MmapOptions::new();

    // SAFETY: mapping a file is unsafe in Rust because another process could
    // change the file while the slice exists. This file is in a directory the
    // program has just created, and nothing else opens it.
    let mut mapping = unsafe {
        if shared {
            options.map_mut(&file)?
        } else {
            options.map_copy(&file)?
        }
    };

    mapping.copy_from_slice(b"HELLO");
    drop(mapping);

    fs::read_to_string(path)
}

/// Compares a private file mapping with a shared one.
fn files() -> io::Result<()> {
    let dir = std::env::temp_dir().join(format!("vmfacts-{}", std::process::id()));
    let path = dir.join("file");

    fs::create_dir_all(&dir)?;

    for (name, shared) in [("private", false), ("shared", true)] {
        let text = through_mapping(&path, shared)?;
        println!("HELLO written through a {name} file mapping: the file holds {text:?}");
    }

    fs::remove_dir_all(&dir)
}

64 pages mapped: 0 resident
after writing to 10 pages: 10 resident
after reading 1 more page: 11 resident, the byte read is 0
permissions of the mapping: rw-p, after mprotect: r--p
HELLO written through a private file mapping: the file holds "hello"
HELLO written through a shared file mapping: the file holds "HELLO"

Modelling an address space

In the Rust version, a frame is an Rc of an array, and the reference count of the Rc is the refs field of the Go version.

Listing 7.11 The imports of vmmodel, in Go. code/go/vmmodel/main.go

import (
	"errors"
	"fmt"
)

Listing 7.12 The imports of vmmodel, in Rust. code/rust/vmmodel/src/main.rs

use std::cell::Cell;
use std::collections::HashMap;
use std::fmt;
use std::rc::Rc;

Listing 7.13 Frames, mappings, entries and the address space, in Go. code/go/vmmodel/main.go

const pageSize = 4096

// errSegv is the result of an access that no mapping allows. The kernel
// sends the process the signal SIGSEGV.
var errSegv = errors.New("SIGSEGV")

// A frame is a page of physical memory. refs counts the page table entries
// that refer to it.
type frame struct {
	data [pageSize]byte
	refs int
}

// memory is the physical memory shared by all address spaces: the number of
// frames allocated, and the frame of zeros that every read of an untouched
// page maps.
type memory struct {
	frames int
	zero   *frame
}

func (m *memory) allocate() *frame {
	m.frames++

	return &frame{refs: 1}
}

// A mapping is a range of virtual pages and the access it permits.
type mapping struct {
	start, pages int
	writable     bool
}

// An entry is a page table entry: the frame behind a virtual page, and
// whether the processor may write to it without a fault.
type entry struct {
	frame    *frame
	writable bool
}

// A space is the address space of a process.
type space struct {
	memory   *memory
	mappings []mapping
	table    map[int]entry
	faults   int
	nextPage int
}

func newSpace() *space {
	return &space{
		memory:   &memory{zero: &frame{refs: 1}},
		table:    map[int]entry{},
		nextPage: 16,
	}
}

Listing 7.14 Frames, mappings, entries and the address space, in Rust. code/rust/vmmodel/src/main.rs

const PAGE_SIZE: usize = 4096;

/// The result of an access that no mapping allows. The kernel sends the
/// process the signal `SIGSEGV`.
#[derive(Debug)]
struct Segv;

impl fmt::Display for Segv {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("SIGSEGV")
    }
}

/// A page of physical memory. The reference count of the `Rc` is the number
/// of page table entries that refer to it.
type Frame = Rc<[u8; PAGE_SIZE]>;

/// The physical memory shared by all address spaces: the number of frames
/// allocated, and the frame of zeros that every read of an untouched page
/// maps.
struct Memory {
    frames: Cell<usize>,
    zero: Frame,
}

impl Memory {
    fn allocate(&self, contents: [u8; PAGE_SIZE]) -> Frame {
        self.frames.set(self.frames.get() + 1);

        Rc::new(contents)
    }
}

/// A range of virtual pages and the access it permits.
#[derive(Clone, Copy)]
struct Mapping {
    start: usize,
    pages: usize,
    writable: bool,
}

/// A page table entry: the frame behind a virtual page, and whether the
/// processor may write to it without a fault.
#[derive(Clone)]
struct Entry {
    frame: Frame,
    writable: bool,
}

/// The address space of a process.
struct Space {
    memory: Rc<Memory>,
    mappings: Vec<Mapping>,
    table: HashMap<usize, Entry>,
    faults: usize,
    next_page: usize,
}

impl Space {
    fn new() -> Space {
        let memory = Memory {
            frames: Cell::new(0),
            zero: Rc::new([0; PAGE_SIZE]),
        };

        Space {
            memory: Rc::new(memory),
            mappings: Vec::new(),
            table: HashMap::new(),
            faults: 0,
            next_page: 16,
        }
    }
}

mmap only records the range.

Listing 7.15 Adding a mapping, in Go. code/go/vmmodel/main.go

// mmap records a new mapping and returns its address. It allocates no frame
// and writes no page table entry.
func (s *space) mmap(pages int, writable bool) int {
	start := s.nextPage
	s.mappings = append(s.mappings, mapping{start: start, pages: pages, writable: writable})
	s.nextPage += pages + 1

	return start * pageSize
}

// find returns the mapping that contains a page.
func (s *space) find(page int) (mapping, bool) {
	for _, m := range s.mappings {
		if page >= m.start && page < m.start+m.pages {
			return m, true
		}
	}

	return mapping{}, false
}

Listing 7.16 Adding a mapping, in Rust. code/rust/vmmodel/src/main.rs

impl Space {
    /// Records a new mapping and returns its address. It allocates no frame
    /// and writes no page table entry.
    fn mmap(&mut self, pages: usize, writable: bool) -> usize {
        let start = self.next_page;

        self.mappings.push(Mapping {
            start,
            pages,
            writable,
        });
        self.next_page += pages + 1;

        start * PAGE_SIZE
    }

    /// Returns the mapping that contains a page.
    fn find(&self, page: usize) -> Option<Mapping> {
        self.mappings
            .iter()
            .copied()
            .find(|mapping| (mapping.start..mapping.start + mapping.pages).contains(&page))
    }
}

read and write play the part of the processor. They consult the page table and call fault when the entry is missing or forbids the access.

Listing 7.17 An access through the page table, in Go. code/go/vmmodel/main.go

// read returns the byte at an address. The processor uses the page table
// entry when one exists, and faults when none does.
func (s *space) read(address int) (byte, error) {
	page, offset := address/pageSize, address%pageSize

	if _, present := s.table[page]; !present {
		if err := s.fault(page, false); err != nil {
			return 0, err
		}
	}

	return s.table[page].frame.data[offset], nil
}

// write stores a byte at an address. It faults when the page has no entry or
// the entry does not permit writing.
func (s *space) write(address int, value byte) error {
	page, offset := address/pageSize, address%pageSize

	if !s.table[page].writable {
		if err := s.fault(page, true); err != nil {
			return err
		}
	}

	s.table[page].frame.data[offset] = value

	return nil
}

Listing 7.18 An access through the page table, in Rust. code/rust/vmmodel/src/main.rs

impl Space {
    /// Returns the byte at an address. The processor uses the page table
    /// entry when one exists, and faults when none does.
    fn read(&mut self, address: usize) -> Result<u8, Segv> {
        let (page, offset) = (address / PAGE_SIZE, address % PAGE_SIZE);

        if !self.table.contains_key(&page) {
            self.fault(page, false)?;
        }

        self.table
            .get(&page)
            .map(|entry| entry.frame[offset])
            .ok_or(Segv)
    }

    /// Stores a byte at an address. It faults when the page has no entry or
    /// the entry does not permit writing.
    fn write(&mut self, address: usize, value: u8) -> Result<(), Segv> {
        let (page, offset) = (address / PAGE_SIZE, address % PAGE_SIZE);

        if !self.table.get(&page).is_some_and(|entry| entry.writable) {
            self.fault(page, true)?;
        }

        let entry = self.table.get_mut(&page).ok_or(Segv)?;

        // A writable entry is the only user of its frame, so `make_mut`
        // returns the frame itself.
        Rc::make_mut(&mut entry.frame)[offset] = value;

        Ok(())
    }
}

fault is the kernel’s handler. After the check against the mappings it has 4 cases.

Case 1 explains the measurement of 11 resident pages. Every read of an untouched anonymous page maps the same frame of zeros, so reading costs no memory. The frame is mapped read-only, and the first write replaces it with a private frame through case 4.

Listing 7.19 The page fault handler, in Go. code/go/vmmodel/main.go

// fault is the kernel's page fault handler. It checks the access against
// the mappings and then repairs the page table entry so that the access can
// be repeated.
func (s *space) fault(page int, write bool) error {
	m, ok := s.find(page)

	if !ok || write && !m.writable {
		return errSegv
	}

	s.faults++
	e, present := s.table[page]

	switch {
	case !write:
		// A read of an untouched page maps the shared zero frame.
		s.memory.zero.refs++
		s.table[page] = entry{frame: s.memory.zero}

	case !present:
		// A write to an untouched page gets a new frame of zeros.
		s.table[page] = entry{frame: s.memory.allocate(), writable: true}

	case e.frame.refs == 1:
		// The frame has no other user. Writing is enabled in place.
		s.table[page] = entry{frame: e.frame, writable: true}

	default:
		// The frame is shared. The writer gets a private copy.
		private := s.memory.allocate()
		private.data = e.frame.data
		e.frame.refs--
		s.table[page] = entry{frame: private, writable: true}
	}

	return nil
}

Listing 7.20 The page fault handler, in Rust. code/rust/vmmodel/src/main.rs

impl Space {
    /// The kernel's page fault handler. It checks the access against the
    /// mappings and then repairs the page table entry so that the access can
    /// be repeated.
    fn fault(&mut self, page: usize, write: bool) -> Result<(), Segv> {
        let mapping = self.find(page).ok_or(Segv)?;

        if write && !mapping.writable {
            return Err(Segv);
        }

        self.faults += 1;

        let entry = match (write, self.table.remove(&page)) {
            // A read of an untouched page maps the shared zero frame.
            (false, _) => Entry {
                frame: Rc::clone(&self.memory.zero),
                writable: false,
            },

            // A write to an untouched page gets a new frame of zeros.
            (true, None) => Entry {
                frame: self.memory.allocate([0; PAGE_SIZE]),
                writable: true,
            },

            // The frame has no other user. Writing is enabled in place.
            (true, Some(entry)) if Rc::strong_count(&entry.frame) == 1 => Entry {
                frame: entry.frame,
                writable: true,
            },

            // The frame is shared. The writer gets a private copy.
            (true, Some(entry)) => Entry {
                frame: self.memory.allocate(*entry.frame),
                writable: true,
            },
        };

        self.table.insert(page, entry);

        Ok(())
    }
}

Copy-on-write

fork gives the child a copy of the parent’s address space. Copying every frame would be slow, and most of the work would be wasted, since the usual next step of the child is exec, which discards the address space. The kernel therefore copies only the page table. Parent and child refer to the same frames, and every entry is made read-only in both.

Listing 7.21 Forking an address space, in Go. code/go/vmmodel/main.go

// fork returns a child with the same mappings and a page table that refers
// to the same frames. Every entry is made read-only in the parent and in the
// child, so that the first write by either causes a fault.
func (s *space) fork() *space {
	child := &space{
		memory:   s.memory,
		mappings: append([]mapping(nil), s.mappings...),
		table:    map[int]entry{},
		nextPage: s.nextPage,
	}

	for page, e := range s.table {
		e.frame.refs++
		e.writable = false
		s.table[page], child.table[page] = e, e
	}

	return child
}

Listing 7.22 Forking an address space, in Rust. code/rust/vmmodel/src/main.rs

impl Space {
    /// Returns a child with the same mappings and a page table that refers to
    /// the same frames. Every entry is made read-only in the parent and in
    /// the child, so that the first write by either causes a fault.
    fn fork(&mut self) -> Space {
        for entry in self.table.values_mut() {
            entry.writable = false;
        }

        Space {
            memory: Rc::clone(&self.memory),
            mappings: self.mappings.clone(),
            table: self.table.clone(),
            faults: 0,
            next_page: self.next_page,
        }
    }
}

Reads continue without faults. The first write to a page by either process faults. The handler sees a read-only entry in a writable mapping, copies the frame for the writer, and the processes diverge on that page only.

A private file mapping uses the same mechanism. Its pages initially refer to the file’s frames, read-only, and a write copies the page. The file is unchanged, as vmfacts measured.

Listing 7.23 The trace, in Go. code/go/vmmodel/main.go

// report prints a step with the fault count of a space and the number of
// frames allocated.
func (s *space) report(step string) {
	fmt.Printf("%-34s faults %d, frames %d\n", step, s.faults, s.memory.frames)
}

func run() error {
	parent := newSpace()
	base := parent.mmap(4, true)
	parent.report("mmap of 4 pages:")

	if err := parent.write(base, 'a'); err != nil {
		return err
	}

	parent.report("write to page 0:")

	value, err := parent.read(base + pageSize)

	if err != nil {
		return err
	}

	parent.report(fmt.Sprintf("read of page 1, value %d:", value))

	if err := parent.write(base+pageSize, 'x'); err != nil {
		return err
	}

	parent.report("write to page 1:")

	child := parent.fork()
	child.report("fork, in the child:")

	if err := child.write(base, 'b'); err != nil {
		return err
	}

	child.report("child writes page 0:")

	if err := parent.write(base, 'c'); err != nil {
		return err
	}

	parent.report("parent writes page 0:")

	inParent, err := parent.read(base)

	if err != nil {
		return err
	}

	inChild, err := child.read(base)

	if err != nil {
		return err
	}

	fmt.Printf("page 0 holds %q in the parent and %q in the child\n", inParent, inChild)

	fmt.Printf("write outside every mapping: %v\n", parent.write(base+100*pageSize, 1))

	readOnly := parent.mmap(1, false)
	fmt.Printf("write to a read-only mapping: %v\n", parent.write(readOnly, 1))

	return nil
}

func main() {
	if err := run(); err != nil {
		fmt.Println(err)
	}
}

Listing 7.24 The trace, in Rust. code/rust/vmmodel/src/main.rs

impl Space {
    /// Prints a step with the fault count of a space and the number of frames
    /// allocated.
    fn report(&self, step: &str) {
        println!(
            "{step:<34} faults {}, frames {}",
            self.faults,
            self.memory.frames.get()
        );
    }
}

fn run() -> Result<(), Segv> {
    let mut parent = Space::new();
    let base = parent.mmap(4, true);
    parent.report("mmap of 4 pages:");

    parent.write(base, b'a')?;
    parent.report("write to page 0:");

    let value = parent.read(base + PAGE_SIZE)?;
    parent.report(&format!("read of page 1, value {value}:"));

    parent.write(base + PAGE_SIZE, b'x')?;
    parent.report("write to page 1:");

    let mut child = parent.fork();
    child.report("fork, in the child:");

    child.write(base, b'b')?;
    child.report("child writes page 0:");

    parent.write(base, b'c')?;
    parent.report("parent writes page 0:");

    println!(
        "page 0 holds {:?} in the parent and {:?} in the child",
        char::from(parent.read(base)?),
        char::from(child.read(base)?)
    );

    if let Err(error) = parent.write(base + 100 * PAGE_SIZE, 1) {
        println!("write outside every mapping: {error}");
    }

    let read_only = parent.mmap(1, false);

    if let Err(error) = parent.write(read_only, 1) {
        println!("write to a read-only mapping: {error}");
    }

    Ok(())
}

fn main() {
    if let Err(error) = run() {
        println!("{error}");
    }
}

mmap of 4 pages:                   faults 0, frames 0
write to page 0:                   faults 1, frames 1
read of page 1, value 0:           faults 2, frames 1
write to page 1:                   faults 3, frames 2
fork, in the child:                faults 0, frames 2
child writes page 0:               faults 1, frames 3
parent writes page 0:              faults 4, frames 3
page 0 holds 'c' in the parent and 'b' in the child
write outside every mapping: SIGSEGV
write to a read-only mapping: SIGSEGV

How much memory a process uses

Since mapping and allocating are separate, a process has 2 sizes.

$ ps -o pid,vsz,rss,min_flt,maj_flt,comm -p 580844
    PID    VSZ   RSS  MINFL  MAJFL COMMAND
 580844  11204   776    114      0 sleep

The sleep process has mapped 11 MB and uses under 1 MB, because most of the pages of its libraries have not been touched. It took 114 page faults to start, each bringing in a single page or a few. RSS is the number to look at when asking how much memory a process costs, with the reservation that pages shared with other processes, such as the code of libc, are counted in the RSS of each.

Because memory is given out at the first access, the kernel can promise more than it has. This is overcommit: the sum of the mappings of all processes may exceed physical memory and swap, on the assumption that not all of it will be touched. The policy is in /proc/sys/vm/overcommit_memory, and the value 1 on this machine means that mmap does not refuse a request for lack of memory. When the promises are called in and no frame is free, the kernel first reclaims frames, by discarding page cache pages (chapter 8) and by writing anonymous pages to swap space on disk. When that is not enough, the OOM killer ends a process to free its memory. Chapter 12 shows it at work inside a memory limit.

The heap and the stack

malloc is a library function, and the kernel knows nothing about individual allocations. The allocator asks the kernel for memory in large pieces and divides them itself. It has 2 ways to ask. brk moves the end of the [heap] mapping, which suits many small allocations. For a large request, 128 KiB or more in the allocator of glibc, it calls mmap for an anonymous mapping of its own, which can be returned to the kernel independently. Both give addresses without frames, so memory that a program allocates and does not touch costs nothing.

The stack is a mapping that the kernel extends on demand. An access directly below its current end causes a page fault, and the handler enlarges the mapping, up to the limit shown by ulimit -s.

Inspecting memory on a live system

The complete programs

Listing 7.25 The complete vmfacts, in Go. code/go/vmfacts/main.go

// Command vmfacts measures how the kernel fills an address space: when a
// mapped page receives physical memory, what an untouched page contains, how
// a mapping's permissions appear in /proc/self/maps, and where a write
// through a private or a shared file mapping ends up.
//
// Run it from code/go with:
//
//	go run ./vmfacts
package main

import (
	"bufio"
	"encoding/binary"
	"fmt"
	"log"
	"os"
	"path/filepath"
	"reflect"
	"strings"

	"golang.org/x/sys/unix"
)


const pageSize = 4096

// address returns the virtual address of the first byte of a mapping.
func address(mapping []byte) uintptr {
	return reflect.ValueOf(mapping).Pointer()
}

// resident counts the pages of a mapping that have physical memory. The
// kernel publishes 8 bytes per virtual page in /proc/self/pagemap, and bit 63
// of an entry is set when the page is present in memory.
func resident(mapping []byte) (int, error) {
	pagemap, err := os.Open("/proc/self/pagemap")

	if err != nil {
		return 0, err
	}

	defer pagemap.Close()

	entries := make([]byte, 8*len(mapping)/pageSize)

	if _, err := pagemap.ReadAt(entries, int64(address(mapping)/pageSize*8)); err != nil {
		return 0, err
	}

	count := 0

	for i := 0; i < len(entries); i += 8 {
		if binary.LittleEndian.Uint64(entries[i:])>>63 == 1 {
			count++
		}
	}

	return count, nil
}


// demand maps 64 pages of anonymous memory, writes to 10 of them, reads 1
// more, and counts the resident pages after each step.
func demand() error {
	mapping, err := unix.Mmap(-1, 0, 64*pageSize, unix.PROT_READ|unix.PROT_WRITE, unix.MAP_PRIVATE|unix.MAP_ANONYMOUS)

	if err != nil {
		return err
	}

	defer unix.Munmap(mapping)

	before, err := resident(mapping)

	if err != nil {
		return err
	}

	for page := range 10 {
		mapping[page*pageSize] = 1
	}

	written, err := resident(mapping)

	if err != nil {
		return err
	}

	value := mapping[20*pageSize]
	read, err := resident(mapping)

	if err != nil {
		return err
	}

	fmt.Printf("64 pages mapped: %d resident\n", before)
	fmt.Printf("after writing to 10 pages: %d resident\n", written)
	fmt.Printf("after reading 1 more page: %d resident, the byte read is %d\n", read, value)

	return nil
}


// permissions returns the permission column of the line in /proc/self/maps
// that starts at the address of a mapping.
func permissions(mapping []byte) (string, error) {
	maps, err := os.Open("/proc/self/maps")

	if err != nil {
		return "", err
	}

	defer maps.Close()

	start := fmt.Sprintf("%x-", address(mapping))

	for lines := bufio.NewScanner(maps); lines.Scan(); {
		if fields := strings.Fields(lines.Text()); strings.HasPrefix(fields[0], start) {
			return fields[1], nil
		}
	}

	return "", fmt.Errorf("mapping at %s not found", start)
}

// protect reads the permissions of a mapping before and after mprotect
// removes the permission to write.
func protect() error {
	mapping, err := unix.Mmap(-1, 0, pageSize, unix.PROT_READ|unix.PROT_WRITE, unix.MAP_PRIVATE|unix.MAP_ANONYMOUS)

	if err != nil {
		return err
	}

	defer unix.Munmap(mapping)

	before, err := permissions(mapping)

	if err != nil {
		return err
	}

	if err := unix.Mprotect(mapping, unix.PROT_READ); err != nil {
		return err
	}

	after, err := permissions(mapping)

	if err != nil {
		return err
	}

	fmt.Printf("permissions of the mapping: %s, after mprotect: %s\n", before, after)

	return nil
}


// throughMapping maps a file that holds "hello", writes "HELLO" through the
// mapping, unmaps it, and returns the contents of the file.
func throughMapping(path string, flags int) (string, error) {
	if err := os.WriteFile(path, []byte("hello"), 0o644); err != nil {
		return "", err
	}

	file, err := os.OpenFile(path, os.O_RDWR, 0)

	if err != nil {
		return "", err
	}

	defer file.Close()

	mapping, err := unix.Mmap(int(file.Fd()), 0, 5, unix.PROT_READ|unix.PROT_WRITE, flags)

	if err != nil {
		return "", err
	}

	copy(mapping, "HELLO")

	if err := unix.Munmap(mapping); err != nil {
		return "", err
	}

	text, err := os.ReadFile(path)

	return string(text), err
}

// files compares a private file mapping with a shared one.
func files() error {
	dir, err := os.MkdirTemp("", "vmfacts")

	if err != nil {
		return err
	}

	defer os.RemoveAll(dir)

	path := filepath.Join(dir, "file")

	for _, kind := range []struct {
		name  string
		flags int
	}{{"private", unix.MAP_PRIVATE}, {"shared", unix.MAP_SHARED}} {
		text, err := throughMapping(path, kind.flags)

		if err != nil {
			return err
		}

		fmt.Printf("HELLO written through a %s file mapping: the file holds %q\n", kind.name, text)
	}

	return nil
}


func main() {
	for _, experiment := range []func() error{demand, protect, files} {
		if err := experiment(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 7.26 The complete vmfacts, in Rust. code/rust/vmfacts/src/main.rs

//! Measures how the kernel fills an address space: when a mapped page
//! receives physical memory, what an untouched page contains, how a mapping's
//! permissions appear in `/proc/self/maps`, and where a write through a
//! private or a shared file mapping ends up.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p vmfacts
//! ```

use std::fs::{self, File, OpenOptions};
use std::io;
use std::os::unix::fs::FileExt;
use std::path::Path;

use memmap2::{MmapMut, MmapOptions};

const PAGE_SIZE: usize = 4096;

/// Counts the pages of a mapping that have physical memory. The kernel
/// publishes 8 bytes per virtual page in `/proc/self/pagemap`, and bit 63 of
/// an entry is set when the page is present in memory.
fn resident(mapping: &[u8]) -> io::Result<usize> {
    let pagemap = File::open("/proc/self/pagemap")?;
    let mut entries = vec![0u8; 8 * mapping.len() / PAGE_SIZE];
    let offset = mapping.as_ptr() as usize / PAGE_SIZE * 8;

    pagemap.read_exact_at(&mut entries, offset as u64)?;

    let (entries, _) = entries.as_chunks::<8>();
    let count = entries.iter().filter(|entry| entry[7] >> 7 == 1).count();

    Ok(count)
}

/// Maps 64 pages of anonymous memory, writes to 10 of them, reads 1 more, and
/// counts the resident pages after each step.
fn demand() -> io::Result<()> {
    let mut mapping = MmapMut::map_anon(64 * PAGE_SIZE)?;

    let before = resident(&mapping)?;

    for page in 0..10 {
        mapping[page * PAGE_SIZE] = 1;
    }

    let written = resident(&mapping)?;

    let value = mapping[20 * PAGE_SIZE];
    let read = resident(&mapping)?;

    println!("64 pages mapped: {before} resident");
    println!("after writing to 10 pages: {written} resident");
    println!("after reading 1 more page: {read} resident, the byte read is {value}");

    Ok(())
}

/// Returns the permission column of the line in `/proc/self/maps` that starts
/// at the address of a mapping.
fn permissions(mapping: &[u8]) -> io::Result<String> {
    let start = format!("{:x}-", mapping.as_ptr() as usize);

    fs::read_to_string("/proc/self/maps")?
        .lines()
        .filter(|line| line.starts_with(&start))
        .find_map(|line| line.split_whitespace().nth(1))
        .map(str::to_owned)
        .ok_or_else(|| io::ErrorKind::NotFound.into())
}

/// Reads the permissions of a mapping before and after mprotect removes the
/// permission to write.
fn protect() -> io::Result<()> {
    let mapping = MmapMut::map_anon(PAGE_SIZE)?;
    let before = permissions(&mapping)?;

    let mapping = mapping.make_read_only()?;
    let after = permissions(&mapping)?;

    println!("permissions of the mapping: {before}, after mprotect: {after}");

    Ok(())
}

/// Maps a file that holds "hello", writes "HELLO" through the mapping, unmaps
/// it, and returns the contents of the file.
fn through_mapping(path: &Path, shared: bool) -> io::Result<String> {
    fs::write(path, "hello")?;

    let file = OpenOptions::new().read(true).write(true).open(path)?;
    let options = MmapOptions::new();

    // SAFETY: mapping a file is unsafe in Rust because another process could
    // change the file while the slice exists. This file is in a directory the
    // program has just created, and nothing else opens it.
    let mut mapping = unsafe {
        if shared {
            options.map_mut(&file)?
        } else {
            options.map_copy(&file)?
        }
    };

    mapping.copy_from_slice(b"HELLO");
    drop(mapping);

    fs::read_to_string(path)
}

/// Compares a private file mapping with a shared one.
fn files() -> io::Result<()> {
    let dir = std::env::temp_dir().join(format!("vmfacts-{}", std::process::id()));
    let path = dir.join("file");

    fs::create_dir_all(&dir)?;

    for (name, shared) in [("private", false), ("shared", true)] {
        let text = through_mapping(&path, shared)?;
        println!("HELLO written through a {name} file mapping: the file holds {text:?}");
    }

    fs::remove_dir_all(&dir)
}

fn main() -> io::Result<()> {
    demand()?;
    protect()?;
    files()
}

Listing 7.27 The complete vmmodel, in Go. code/go/vmmodel/main.go

// Command vmmodel models an address space: a list of mappings, a page table
// that is filled on demand by page faults, a shared zero page, and
// copy-on-write after fork.
//
// Run it from code/go with:
//
//	go run ./vmmodel
package main

import (
	"errors"
	"fmt"
)


const pageSize = 4096

// errSegv is the result of an access that no mapping allows. The kernel
// sends the process the signal SIGSEGV.
var errSegv = errors.New("SIGSEGV")

// A frame is a page of physical memory. refs counts the page table entries
// that refer to it.
type frame struct {
	data [pageSize]byte
	refs int
}

// memory is the physical memory shared by all address spaces: the number of
// frames allocated, and the frame of zeros that every read of an untouched
// page maps.
type memory struct {
	frames int
	zero   *frame
}

func (m *memory) allocate() *frame {
	m.frames++

	return &frame{refs: 1}
}

// A mapping is a range of virtual pages and the access it permits.
type mapping struct {
	start, pages int
	writable     bool
}

// An entry is a page table entry: the frame behind a virtual page, and
// whether the processor may write to it without a fault.
type entry struct {
	frame    *frame
	writable bool
}

// A space is the address space of a process.
type space struct {
	memory   *memory
	mappings []mapping
	table    map[int]entry
	faults   int
	nextPage int
}

func newSpace() *space {
	return &space{
		memory:   &memory{zero: &frame{refs: 1}},
		table:    map[int]entry{},
		nextPage: 16,
	}
}


// mmap records a new mapping and returns its address. It allocates no frame
// and writes no page table entry.
func (s *space) mmap(pages int, writable bool) int {
	start := s.nextPage
	s.mappings = append(s.mappings, mapping{start: start, pages: pages, writable: writable})
	s.nextPage += pages + 1

	return start * pageSize
}

// find returns the mapping that contains a page.
func (s *space) find(page int) (mapping, bool) {
	for _, m := range s.mappings {
		if page >= m.start && page < m.start+m.pages {
			return m, true
		}
	}

	return mapping{}, false
}


// read returns the byte at an address. The processor uses the page table
// entry when one exists, and faults when none does.
func (s *space) read(address int) (byte, error) {
	page, offset := address/pageSize, address%pageSize

	if _, present := s.table[page]; !present {
		if err := s.fault(page, false); err != nil {
			return 0, err
		}
	}

	return s.table[page].frame.data[offset], nil
}

// write stores a byte at an address. It faults when the page has no entry or
// the entry does not permit writing.
func (s *space) write(address int, value byte) error {
	page, offset := address/pageSize, address%pageSize

	if !s.table[page].writable {
		if err := s.fault(page, true); err != nil {
			return err
		}
	}

	s.table[page].frame.data[offset] = value

	return nil
}


// fault is the kernel's page fault handler. It checks the access against
// the mappings and then repairs the page table entry so that the access can
// be repeated.
func (s *space) fault(page int, write bool) error {
	m, ok := s.find(page)

	if !ok || write && !m.writable {
		return errSegv
	}

	s.faults++
	e, present := s.table[page]

	switch {
	case !write:
		// A read of an untouched page maps the shared zero frame.
		s.memory.zero.refs++
		s.table[page] = entry{frame: s.memory.zero}

	case !present:
		// A write to an untouched page gets a new frame of zeros.
		s.table[page] = entry{frame: s.memory.allocate(), writable: true}

	case e.frame.refs == 1:
		// The frame has no other user. Writing is enabled in place.
		s.table[page] = entry{frame: e.frame, writable: true}

	default:
		// The frame is shared. The writer gets a private copy.
		private := s.memory.allocate()
		private.data = e.frame.data
		e.frame.refs--
		s.table[page] = entry{frame: private, writable: true}
	}

	return nil
}


// fork returns a child with the same mappings and a page table that refers
// to the same frames. Every entry is made read-only in the parent and in the
// child, so that the first write by either causes a fault.
func (s *space) fork() *space {
	child := &space{
		memory:   s.memory,
		mappings: append([]mapping(nil), s.mappings...),
		table:    map[int]entry{},
		nextPage: s.nextPage,
	}

	for page, e := range s.table {
		e.frame.refs++
		e.writable = false
		s.table[page], child.table[page] = e, e
	}

	return child
}


// report prints a step with the fault count of a space and the number of
// frames allocated.
func (s *space) report(step string) {
	fmt.Printf("%-34s faults %d, frames %d\n", step, s.faults, s.memory.frames)
}

func run() error {
	parent := newSpace()
	base := parent.mmap(4, true)
	parent.report("mmap of 4 pages:")

	if err := parent.write(base, 'a'); err != nil {
		return err
	}

	parent.report("write to page 0:")

	value, err := parent.read(base + pageSize)

	if err != nil {
		return err
	}

	parent.report(fmt.Sprintf("read of page 1, value %d:", value))

	if err := parent.write(base+pageSize, 'x'); err != nil {
		return err
	}

	parent.report("write to page 1:")

	child := parent.fork()
	child.report("fork, in the child:")

	if err := child.write(base, 'b'); err != nil {
		return err
	}

	child.report("child writes page 0:")

	if err := parent.write(base, 'c'); err != nil {
		return err
	}

	parent.report("parent writes page 0:")

	inParent, err := parent.read(base)

	if err != nil {
		return err
	}

	inChild, err := child.read(base)

	if err != nil {
		return err
	}

	fmt.Printf("page 0 holds %q in the parent and %q in the child\n", inParent, inChild)

	fmt.Printf("write outside every mapping: %v\n", parent.write(base+100*pageSize, 1))

	readOnly := parent.mmap(1, false)
	fmt.Printf("write to a read-only mapping: %v\n", parent.write(readOnly, 1))

	return nil
}

func main() {
	if err := run(); err != nil {
		fmt.Println(err)
	}
}

Listing 7.28 The complete vmmodel, in Rust. code/rust/vmmodel/src/main.rs

//! Models an address space: a list of mappings, a page table that is filled
//! on demand by page faults, a shared zero page, and copy-on-write after
//! fork.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p vmmodel
//! ```

use std::cell::Cell;
use std::collections::HashMap;
use std::fmt;
use std::rc::Rc;

const PAGE_SIZE: usize = 4096;

/// The result of an access that no mapping allows. The kernel sends the
/// process the signal `SIGSEGV`.
#[derive(Debug)]
struct Segv;

impl fmt::Display for Segv {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("SIGSEGV")
    }
}

/// A page of physical memory. The reference count of the `Rc` is the number
/// of page table entries that refer to it.
type Frame = Rc<[u8; PAGE_SIZE]>;

/// The physical memory shared by all address spaces: the number of frames
/// allocated, and the frame of zeros that every read of an untouched page
/// maps.
struct Memory {
    frames: Cell<usize>,
    zero: Frame,
}

impl Memory {
    fn allocate(&self, contents: [u8; PAGE_SIZE]) -> Frame {
        self.frames.set(self.frames.get() + 1);

        Rc::new(contents)
    }
}

/// A range of virtual pages and the access it permits.
#[derive(Clone, Copy)]
struct Mapping {
    start: usize,
    pages: usize,
    writable: bool,
}

/// A page table entry: the frame behind a virtual page, and whether the
/// processor may write to it without a fault.
#[derive(Clone)]
struct Entry {
    frame: Frame,
    writable: bool,
}

/// The address space of a process.
struct Space {
    memory: Rc<Memory>,
    mappings: Vec<Mapping>,
    table: HashMap<usize, Entry>,
    faults: usize,
    next_page: usize,
}

impl Space {
    fn new() -> Space {
        let memory = Memory {
            frames: Cell::new(0),
            zero: Rc::new([0; PAGE_SIZE]),
        };

        Space {
            memory: Rc::new(memory),
            mappings: Vec::new(),
            table: HashMap::new(),
            faults: 0,
            next_page: 16,
        }
    }
}

impl Space {
    /// Records a new mapping and returns its address. It allocates no frame
    /// and writes no page table entry.
    fn mmap(&mut self, pages: usize, writable: bool) -> usize {
        let start = self.next_page;

        self.mappings.push(Mapping {
            start,
            pages,
            writable,
        });
        self.next_page += pages + 1;

        start * PAGE_SIZE
    }

    /// Returns the mapping that contains a page.
    fn find(&self, page: usize) -> Option<Mapping> {
        self.mappings
            .iter()
            .copied()
            .find(|mapping| (mapping.start..mapping.start + mapping.pages).contains(&page))
    }
}

impl Space {
    /// Returns the byte at an address. The processor uses the page table
    /// entry when one exists, and faults when none does.
    fn read(&mut self, address: usize) -> Result<u8, Segv> {
        let (page, offset) = (address / PAGE_SIZE, address % PAGE_SIZE);

        if !self.table.contains_key(&page) {
            self.fault(page, false)?;
        }

        self.table
            .get(&page)
            .map(|entry| entry.frame[offset])
            .ok_or(Segv)
    }

    /// Stores a byte at an address. It faults when the page has no entry or
    /// the entry does not permit writing.
    fn write(&mut self, address: usize, value: u8) -> Result<(), Segv> {
        let (page, offset) = (address / PAGE_SIZE, address % PAGE_SIZE);

        if !self.table.get(&page).is_some_and(|entry| entry.writable) {
            self.fault(page, true)?;
        }

        let entry = self.table.get_mut(&page).ok_or(Segv)?;

        // A writable entry is the only user of its frame, so `make_mut`
        // returns the frame itself.
        Rc::make_mut(&mut entry.frame)[offset] = value;

        Ok(())
    }
}

impl Space {
    /// The kernel's page fault handler. It checks the access against the
    /// mappings and then repairs the page table entry so that the access can
    /// be repeated.
    fn fault(&mut self, page: usize, write: bool) -> Result<(), Segv> {
        let mapping = self.find(page).ok_or(Segv)?;

        if write && !mapping.writable {
            return Err(Segv);
        }

        self.faults += 1;

        let entry = match (write, self.table.remove(&page)) {
            // A read of an untouched page maps the shared zero frame.
            (false, _) => Entry {
                frame: Rc::clone(&self.memory.zero),
                writable: false,
            },

            // A write to an untouched page gets a new frame of zeros.
            (true, None) => Entry {
                frame: self.memory.allocate([0; PAGE_SIZE]),
                writable: true,
            },

            // The frame has no other user. Writing is enabled in place.
            (true, Some(entry)) if Rc::strong_count(&entry.frame) == 1 => Entry {
                frame: entry.frame,
                writable: true,
            },

            // The frame is shared. The writer gets a private copy.
            (true, Some(entry)) => Entry {
                frame: self.memory.allocate(*entry.frame),
                writable: true,
            },
        };

        self.table.insert(page, entry);

        Ok(())
    }
}

impl Space {
    /// Returns a child with the same mappings and a page table that refers to
    /// the same frames. Every entry is made read-only in the parent and in
    /// the child, so that the first write by either causes a fault.
    fn fork(&mut self) -> Space {
        for entry in self.table.values_mut() {
            entry.writable = false;
        }

        Space {
            memory: Rc::clone(&self.memory),
            mappings: self.mappings.clone(),
            table: self.table.clone(),
            faults: 0,
            next_page: self.next_page,
        }
    }
}

impl Space {
    /// Prints a step with the fault count of a space and the number of frames
    /// allocated.
    fn report(&self, step: &str) {
        println!(
            "{step:<34} faults {}, frames {}",
            self.faults,
            self.memory.frames.get()
        );
    }
}

fn run() -> Result<(), Segv> {
    let mut parent = Space::new();
    let base = parent.mmap(4, true);
    parent.report("mmap of 4 pages:");

    parent.write(base, b'a')?;
    parent.report("write to page 0:");

    let value = parent.read(base + PAGE_SIZE)?;
    parent.report(&format!("read of page 1, value {value}:"));

    parent.write(base + PAGE_SIZE, b'x')?;
    parent.report("write to page 1:");

    let mut child = parent.fork();
    child.report("fork, in the child:");

    child.write(base, b'b')?;
    child.report("child writes page 0:");

    parent.write(base, b'c')?;
    parent.report("parent writes page 0:");

    println!(
        "page 0 holds {:?} in the parent and {:?} in the child",
        char::from(parent.read(base)?),
        char::from(child.read(base)?)
    );

    if let Err(error) = parent.write(base + 100 * PAGE_SIZE, 1) {
        println!("write outside every mapping: {error}");
    }

    let read_only = parent.mmap(1, false);

    if let Err(error) = parent.write(read_only, 1) {
        println!("write to a read-only mapping: {error}");
    }

    Ok(())
}

fn main() {
    if let Err(error) = run() {
        println!("{error}");
    }
}

References

• mmap(2), mprotect(2) and proc(5) for pagemap and maps, Linux manual pages: https://man7.org/linux/man-pages/man2/mmap.2.html
• Examining Process Page Tables, Linux kernel documentation: https://www.kernel.org/doc/html/v5.15/admin-guide/mm/pagemap.html
• M. Gorman, Understanding the Linux Virtual Memory Manager, Prentice Hall, 2004.
• mm/memory.c and mm/mmap.c, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/mm/memory.c

The Page Cache

The page cache is the kernel’s in-memory copy of file contents. It holds frames of file data, indexed by file and by page number within the file. read copies from these frames and write copies into them. The disk is involved only when a page is missing from the cache, or when a changed page is written back.

In this chapter, we will learn when file data moves between the cache and the disk. We will first measure with cachefacts how many pages of a file are cached after writes and reads of different kinds. Then we will write pagecache, a model with readahead, dirty pages and eviction, and compare its behaviour with the measurements.

Why a cache

A disk request takes far longer than a copy in memory: roughly 100 microseconds for a fast solid-state drive against well under 1 microsecond for a page that is already in memory. The kernel therefore keeps file pages in frames that no process currently needs and serves file operations from them.

A dirty page is newer than the disk. The kernel writes dirty pages to the disk in the background, a process called writeback, typically within 30 seconds. write therefore returns before the data is on the disk, and data written shortly before a power failure can be lost. A program that needs the data to be durable calls fsync, which returns when the file’s dirty pages have reached the disk.

The cache uses memory that is otherwise free, and gives it up when processes need it. A clean page can be discarded at any time, because the disk has the same contents. A dirty page has to be written first.

One read, step by step

A process calls read(fd, buf, 4096) on a file whose offset is 8192.

1. The kernel finds the struct file for the descriptor and, through it, the inode of the file (chapter 2).
2. It divides the offset by the page size. Offset 8192 is the start of page number 2 of the file.
3. It looks page 2 up in the cache of this file. Each inode has its own index of cached pages.
4. If the page is there, the kernel copies 4096 bytes from it to buf, advances the offset, and returns. No request reaches the disk.
5. If the page is missing, the kernel allocates a frame, asks the file system to read the block from the disk into it, and inserts the frame in the index. The process sleeps until the disk has answered. Then step 4 follows.

A read that finds its page is a cache hit, and one that has to go to the disk is a cache miss. The page fault of chapter 7 that reads a page of a mapped file takes the same path from step 3 on, and the frame it maps is the frame in the cache. read and mmap therefore see the same copy of a file.

The kernel’s structure

The index of cached pages of a file is a struct address_space, embedded in the inode and reached through inode->i_mapping. This is the definition in include/linux/fs.h of Linux 5.15, reduced to the fields we use.

struct address_space {
        struct inode *host;                    /* the file these pages belong to */
        struct xarray i_pages;                 /* page number -> struct page */
        unsigned long nrpages;                 /* number of cached pages */
        pgoff_t writeback_index;               /* where writeback continues */
        const struct address_space_operations *a_ops;  /* read and write a page */
        /* ... */
};

Each cached frame is a struct page (chapter 7) whose mapping field points back to this address_space and whose index is its page number. Bits in its flags record its state: PG_uptodate when its contents are valid, PG_dirty when it has been written and the disk has not, and PG_writeback while it is being written to the disk.

How much memory is cache

The page cache uses whatever memory nothing else needs. On our machine, with 16 GB of memory:

$ free -m
               total        used        free      shared  buff/cache   available
Mem:           15655        8881         300         294        6473        6197
$ grep -E 'Cached|Dirty|Writeback:' /proc/meminfo
Cached:          5760296 kB
Dirty:              2488 kB
Writeback:             0 kB

Only 300 MB is free, and that is the normal state of a healthy system. 6473 MB is buff/cache, mostly page cache, and the kernel can take it back at any moment, which is why the available column, 6197 MB, is the number that says how much memory a new program could get. Dirty is the part of the cache that has not been written to disk yet, 2.5 MB at that moment.

Measuring the cache

cachefacts counts the cached pages of a file with the fincore utility from util-linux. It uses golang.org/x/sys/unix in Go and the nix crate with the fs feature in Rust, for posix_fadvise.

$ cargo new cachefacts
$ cargo add -p cachefacts nix --features fs

The file must be on a disk-backed file system for the measurement to mean anything. On this machine /tmp is on ext4. drop makes the file’s pages clean with fsync and then asks the kernel to discard them with the advice POSIX_FADV_DONTNEED.

Listing 8.1 The imports of cachefacts, in Go. code/go/cachefacts/main.go

import (
	"fmt"
	"io"
	"log"
	"os"
	"os/exec"
	"path/filepath"
	"strconv"
	"strings"

	"golang.org/x/sys/unix"
)

Listing 8.2 The imports of cachefacts, in Rust. code/rust/cachefacts/src/main.rs

use std::fs::{self, File, OpenOptions};
use std::io::{self, Write};
use std::os::unix::fs::FileExt;
use std::path::Path;
use std::process::Command;

use nix::fcntl::{PosixFadviseAdvice, posix_fadvise};

Listing 8.3 Counting and dropping the cached pages of a file, in Go. code/go/cachefacts/main.go

const pageSize = 4096

// cached returns the number of pages of a file that are in the page cache,
// as reported by the fincore utility.
func cached(path string) (int, error) {
	text, err := exec.Command("fincore", "--noheadings", "--output", "PAGES", path).Output()

	if err != nil {
		return 0, err
	}

	return strconv.Atoi(strings.TrimSpace(string(text)))
}

// report prints a step and the number of cached pages after it.
func report(step, path string) error {
	pages, err := cached(path)

	if err != nil {
		return err
	}

	fmt.Printf("%-38s cached pages: %d\n", step+":", pages)

	return nil
}

// drop writes the file's dirty pages to disk and asks the kernel to remove
// the file's pages from the cache.
func drop(file *os.File) error {
	if err := file.Sync(); err != nil {
		return err
	}

	return unix.Fadvise(int(file.Fd()), 0, 0, unix.FADV_DONTNEED)
}

Listing 8.4 Counting and dropping the cached pages of a file, in Rust. code/rust/cachefacts/src/main.rs

const PAGE_SIZE: usize = 4096;

/// Returns the number of pages of a file that are in the page cache, as
/// reported by the fincore utility.
fn cached(path: &Path) -> io::Result<usize> {
    let output = Command::new("fincore")
        .args(["--noheadings", "--output", "PAGES"])
        .arg(path)
        .output()?;

    String::from_utf8_lossy(&output.stdout)
        .trim()
        .parse()
        .map_err(|_| io::ErrorKind::InvalidData.into())
}

/// Prints a step and the number of cached pages after it.
fn report(step: &str, path: &Path) -> io::Result<()> {
    let pages = cached(path)?;
    println!("{:<38} cached pages: {pages}", format!("{step}:"));

    Ok(())
}

/// Writes the file's dirty pages to disk and asks the kernel to remove the
/// file's pages from the cache.
fn drop_pages(file: &File) -> io::Result<()> {
    file.sync_all()?;
    posix_fadvise(file, 0, 0, PosixFadviseAdvice::POSIX_FADV_DONTNEED)?;

    Ok(())
}

run writes 64 pages, drops them, reads 1 byte, reads the whole file, drops the pages again, and reads 1 byte after giving the advice POSIX_FADV_RANDOM.

Listing 8.5 Writes and reads of a file of 64 pages, in Go. code/go/cachefacts/main.go

// run writes a file of 64 pages and reads it in 3 ways, dropping the cached
// pages between the steps.
func run(path string) error {
	file, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE, 0o644)

	if err != nil {
		return err
	}

	defer file.Close()

	if _, err := file.Write(make([]byte, 64*pageSize)); err != nil {
		return err
	}

	if err := report("write of 64 pages", path); err != nil {
		return err
	}

	if err := drop(file); err != nil {
		return err
	}

	if err := report("fsync, then FADV_DONTNEED", path); err != nil {
		return err
	}

	one := make([]byte, 1)

	if _, err := file.ReadAt(one, 0); err != nil {
		return err
	}

	if err := report("read of 1 byte at offset 0", path); err != nil {
		return err
	}

	if _, err := io.Copy(io.Discard, io.NewSectionReader(file, 0, 64*pageSize)); err != nil {
		return err
	}

	if err := report("read of the whole file", path); err != nil {
		return err
	}

	if err := drop(file); err != nil {
		return err
	}

	if err := unix.Fadvise(int(file.Fd()), 0, 0, unix.FADV_RANDOM); err != nil {
		return err
	}

	if _, err := file.ReadAt(one, 0); err != nil {
		return err
	}

	return report("dropped, FADV_RANDOM, read of 1 byte", path)
}

Listing 8.6 Writes and reads of a file of 64 pages, in Rust. code/rust/cachefacts/src/main.rs

/// Writes a file of 64 pages and reads it in 3 ways, dropping the cached
/// pages between the steps.
fn run(path: &Path) -> io::Result<()> {
    let mut file = OpenOptions::new()
        .read(true)
        .write(true)
        .create(true)
        .truncate(true)
        .open(path)?;

    file.write_all(&vec![0u8; 64 * PAGE_SIZE])?;
    report("write of 64 pages", path)?;

    drop_pages(&file)?;
    report("fsync, then FADV_DONTNEED", path)?;

    let mut one = [0u8; 1];

    file.read_exact_at(&mut one, 0)?;
    report("read of 1 byte at offset 0", path)?;

    let mut whole = vec![0u8; 64 * PAGE_SIZE];

    file.read_exact_at(&mut whole, 0)?;
    report("read of the whole file", path)?;

    drop_pages(&file)?;
    posix_fadvise(&file, 0, 0, PosixFadviseAdvice::POSIX_FADV_RANDOM)?;

    file.read_exact_at(&mut one, 0)?;
    report("dropped, FADV_RANDOM, read of 1 byte", path)
}

write of 64 pages:                     cached pages: 64
fsync, then FADV_DONTNEED:             cached pages: 0
read of 1 byte at offset 0:            cached pages: 4
read of the whole file:                cached pages: 64
dropped, FADV_RANDOM, read of 1 byte:  cached pages: 1

Readahead

Most files are read from start to end. A disk serves 1 large request much faster than many small ones, so the kernel assumes sequential access and reads more pages than were asked for. This is readahead. The first read at the start of a file loads a small window, 4 pages in the measurement. While the reads continue in order, each window is double the size of the last, up to a limit, which is 32 pages on this machine (/sys/block/<device>/queue/read_ahead_kb is 128). A read at an unrelated position starts again with a small window.

A program that knows its access pattern can say so with posix_fadvise. POSIX_FADV_RANDOM turns readahead off for the descriptor, and POSIX_FADV_SEQUENTIAL doubles the limit.

Modelling the cache

The model reduces the contents of a page to 1 byte and counts requests to the disk.

Listing 8.7 The imports of pagecache, in Go. code/go/pagecache/main.go

import (
	"fmt"
	"slices"
)

Listing 8.8 The imports of pagecache, in Rust. code/rust/pagecache/src/main.rs

use std::collections::BTreeMap;

Listing 8.9 The disk, a cached page, and the cache, in Go. code/go/pagecache/main.go

// maxWindow is the largest number of pages one disk request reads ahead.
const maxWindow = 8

// A disk holds the pages of a file and counts the requests made to it. The
// contents of a page are reduced to one byte.
type disk struct {
	pages    []byte
	requests int
	written  int
}

// A page is a cached copy of a disk page. dirty is set when the copy has
// been written and the disk has not. used is the time of the last access.
type page struct {
	value byte
	dirty bool
	used  int
}

// A cache maps page numbers to cached pages. next is the page that a
// sequential reader would ask for next, and window is the current number of
// pages to read ahead.
type cache struct {
	disk     *disk
	pages    map[int]*page
	capacity int
	clock    int
	next     int
	window   int
}

func newCache(d *disk, capacity int) *cache {
	return &cache{disk: d, pages: map[int]*page{}, capacity: capacity}
}

Listing 8.10 The disk, a cached page, and the cache, in Rust. code/rust/pagecache/src/main.rs

/// The largest number of pages one disk request reads ahead.
const MAX_WINDOW: usize = 8;

/// Holds the pages of a file and counts the requests made to it. The contents
/// of a page are reduced to one byte.
struct Disk {
    pages: Vec<u8>,
    requests: usize,
    written: usize,
}

impl Disk {
    fn new() -> Disk {
        Disk {
            pages: vec![b'-'; 16],
            requests: 0,
            written: 0,
        }
    }
}

/// A cached copy of a disk page. `dirty` is set when the copy has been
/// written and the disk has not. `used` is the time of the last access.
#[derive(Default)]
struct Page {
    value: u8,
    dirty: bool,
    used: usize,
}

/// Maps page numbers to cached pages. `next` is the page that a sequential
/// reader would ask for next, and `window` is the current number of pages to
/// read ahead.
struct Cache {
    disk: Disk,
    pages: BTreeMap<usize, Page>,
    capacity: usize,
    clock: usize,
    next: usize,
    window: usize,
}

impl Cache {
    fn new(capacity: usize) -> Cache {
        Cache {
            disk: Disk::new(),
            pages: BTreeMap::new(),
            capacity,
            clock: 0,
            next: 0,
            window: 0,
        }
    }
}

read implements the lookup and the readahead rule. next is the page number that would follow the previous read. A miss at that position doubles the window, and a miss anywhere else sets it to 1 page.

Listing 8.11 Reading through the cache, with readahead, in Go. code/go/pagecache/main.go

// read returns the contents of a page. A page that is cached costs no disk
// request. For a page that is missing, read makes one request for a window
// of pages. The window doubles while the reads are sequential and returns
// to 1 page when they are not.
func (c *cache) read(index int) byte {
	if _, hit := c.pages[index]; !hit {
		if index == c.next {
			c.window = min(2*max(c.window, 1), maxWindow)
		} else {
			c.window = 1
		}

		c.load(index, c.window)
	}

	c.next = index + 1

	return c.touch(index).value
}

// load reads count pages starting at index from the disk in one request.
func (c *cache) load(index, count int) {
	c.disk.requests++

	for i := index; i < min(index+count, len(c.disk.pages)); i++ {
		if _, hit := c.pages[i]; !hit {
			c.insert(i, &page{value: c.disk.pages[i]})
		}
	}
}

// touch records an access to a cached page and returns the page.
func (c *cache) touch(index int) *page {
	c.clock++
	c.pages[index].used = c.clock

	return c.pages[index]
}

Listing 8.12 Reading through the cache, with readahead, in Rust. code/rust/pagecache/src/main.rs

impl Cache {
    /// Returns the contents of a page. A page that is cached costs no disk
    /// request. For a page that is missing, `read` makes one request for a
    /// window of pages. The window doubles while the reads are sequential and
    /// returns to 1 page when they are not.
    fn read(&mut self, index: usize) -> u8 {
        if !self.pages.contains_key(&index) {
            if index == self.next {
                self.window = (2 * self.window.max(1)).min(MAX_WINDOW);
            } else {
                self.window = 1;
            }

            self.load(index, self.window);
        }

        self.next = index + 1;

        self.touch(index).value
    }

    /// Reads `count` pages starting at `index` from the disk in one request.
    fn load(&mut self, index: usize, count: usize) {
        self.disk.requests += 1;

        for i in index..(index + count).min(self.disk.pages.len()) {
            if !self.pages.contains_key(&i) {
                let value = self.disk.pages[i];

                self.insert(i, value);
            }
        }
    }

    /// Records an access to a page and returns the page. A page that is not
    /// cached is created empty.
    fn touch(&mut self, index: usize) -> &mut Page {
        self.clock += 1;

        let page = self.pages.entry(index).or_default();
        page.used = self.clock;

        page
    }
}

write touches only the cache. sync is the model’s fsync.

Listing 8.13 Writing, and writing back, in Go. code/go/pagecache/main.go

// write changes a page in the cache and marks it dirty. It makes no disk
// request.
func (c *cache) write(index int, value byte) {
	if _, hit := c.pages[index]; !hit {
		c.insert(index, &page{})
	}

	p := c.touch(index)
	p.value, p.dirty = value, true
}

// sync writes every dirty page to the disk and returns the number written.
func (c *cache) sync() int {
	before := c.disk.written

	for index, p := range c.pages {
		c.clean(index, p)
	}

	return c.disk.written - before
}

// clean writes one page to the disk if it is dirty.
func (c *cache) clean(index int, p *page) {
	if p.dirty {
		c.disk.pages[index] = p.value
		c.disk.written++
		p.dirty = false
	}
}

Listing 8.14 Writing, and writing back, in Rust. code/rust/pagecache/src/main.rs

impl Cache {
    /// Changes a page in the cache and marks it dirty. It makes no disk
    /// request.
    fn write(&mut self, index: usize, value: u8) {
        if !self.pages.contains_key(&index) {
            self.insert(index, 0);
        }

        let page = self.touch(index);
        page.value = value;
        page.dirty = true;
    }

    /// Writes every dirty page to the disk and returns the number written.
    fn sync(&mut self) -> usize {
        let before = self.disk.written;

        for (&index, page) in &mut self.pages {
            clean(&mut self.disk, index, page);
        }

        self.disk.written - before
    }
}

/// Writes one page to the disk if it is dirty.
fn clean(disk: &mut Disk, index: usize, page: &mut Page) {
    if page.dirty {
        disk.pages[index] = page.value;
        disk.written += 1;
        page.dirty = false;
    }
}

insert enforces the capacity. It evicts the page with the oldest access time, which is the least recently used policy: a page that has not been used for the longest time is the least likely to be needed soon. The kernel approximates this policy with 2 lists of pages, active and inactive, and avoids keeping an exact access time for each page.

Listing 8.15 Inserting a page, and evicting the least recently used, in Go. code/go/pagecache/main.go

// insert adds a page to the cache. When the cache is full it first evicts
// the page whose last access is the oldest. A dirty page is written to the
// disk before it is removed.
func (c *cache) insert(index int, p *page) {
	if len(c.pages) == c.capacity {
		oldest := -1

		for i, candidate := range c.pages {
			if oldest == -1 || candidate.used < c.pages[oldest].used {
				oldest = i
			}
		}

		c.clean(oldest, c.pages[oldest])
		delete(c.pages, oldest)
	}

	c.clock++
	p.used = c.clock
	c.pages[index] = p
}

// held returns the numbers of the cached pages in increasing order.
func (c *cache) held() []int {
	held := make([]int, 0, len(c.pages))

	for index := range c.pages {
		held = append(held, index)
	}

	slices.Sort(held)

	return held
}

Listing 8.16 Inserting a page, and evicting the least recently used, in Rust. code/rust/pagecache/src/main.rs

impl Cache {
    /// Adds a page to the cache. When the cache is full it first evicts the
    /// page whose last access is the oldest. A dirty page is written to the
    /// disk before it is removed.
    fn insert(&mut self, index: usize, value: u8) {
        if self.pages.len() == self.capacity {
            let oldest = self
                .pages
                .iter()
                .min_by_key(|(_, page)| page.used)
                .map(|(&index, _)| index);

            if let Some(oldest) = oldest
                && let Some(mut page) = self.pages.remove(&oldest)
            {
                clean(&mut self.disk, oldest, &mut page);
            }
        }

        self.clock += 1;

        let page = Page {
            value,
            dirty: false,
            used: self.clock,
        };

        self.pages.insert(index, page);
    }

    /// Returns the numbers of the cached pages in increasing order.
    fn held(&self) -> Vec<usize> {
        self.pages.keys().copied().collect()
    }
}

Listing 8.17 The trace, in Go. code/go/pagecache/main.go

// requests reads the given pages through a new cache and returns the number
// of disk requests.
func requests(order []int) int {
	d := &disk{pages: make([]byte, 16)}
	c := newCache(d, 32)

	for _, index := range order {
		c.read(index)
	}

	return d.requests
}

func main() {
	forward := make([]int, 16)

	for i := range forward {
		forward[i] = i
	}

	backward := slices.Clone(forward)
	slices.Reverse(backward)

	fmt.Printf("16 pages read in order: %d disk requests\n", requests(forward))
	fmt.Printf("16 pages read in reverse order: %d disk requests\n", requests(backward))
	fmt.Printf("16 pages read in order, 2 times: %d disk requests\n", requests(append(forward, forward...)))

	d := &disk{pages: []byte("----------------")}
	c := newCache(d, 4)

	c.write(3, 'x')
	fmt.Printf("write of x to page 3: %d pages written, the disk holds %c\n", d.written, d.pages[3])
	fmt.Printf("sync: %d page written, the disk holds %c\n", c.sync(), d.pages[3])

	for _, index := range []int{2, 1, 0, 3} {
		c.read(index)
	}

	c.read(8)
	fmt.Printf("capacity 4, pages 3 2 1 0 3 used, then page 8: cache holds %v\n", c.held())

	c.write(1, 'y')

	for _, index := range []int{0, 3, 8, 12} {
		c.read(index)
	}

	fmt.Printf("write of y to page 1, then pages 0 3 8 12 used: cache holds %v, the disk holds %c\n", c.held(), d.pages[1])
}

Listing 8.18 The trace, in Rust. code/rust/pagecache/src/main.rs

/// Reads the given pages through a new cache and returns the number of disk
/// requests.
fn requests(order: impl Iterator<Item = usize>) -> usize {
    let mut cache = Cache::new(32);

    for index in order {
        cache.read(index);
    }

    cache.disk.requests
}

/// Formats page numbers the way Go prints a slice.
fn list(pages: &[usize]) -> String {
    let words: Vec<String> = pages.iter().map(usize::to_string).collect();

    format!("[{}]", words.join(" "))
}

fn main() {
    println!("16 pages read in order: {} disk requests", requests(0..16));
    println!(
        "16 pages read in reverse order: {} disk requests",
        requests((0..16).rev())
    );
    println!(
        "16 pages read in order, 2 times: {} disk requests",
        requests((0..16).chain(0..16))
    );

    let mut cache = Cache::new(4);

    cache.write(3, b'x');
    println!(
        "write of x to page 3: {} pages written, the disk holds {}",
        cache.disk.written,
        char::from(cache.disk.pages[3])
    );

    let written = cache.sync();
    println!(
        "sync: {written} page written, the disk holds {}",
        char::from(cache.disk.pages[3])
    );

    for index in [2, 1, 0, 3] {
        cache.read(index);
    }

    cache.read(8);
    println!(
        "capacity 4, pages 3 2 1 0 3 used, then page 8: cache holds {}",
        list(&cache.held())
    );

    cache.write(1, b'y');

    for index in [0, 3, 8, 12] {
        cache.read(index);
    }

    println!(
        "write of y to page 1, then pages 0 3 8 12 used: cache holds {}, the disk holds {}",
        list(&cache.held()),
        char::from(cache.disk.pages[1])
    );
}

16 pages read in order: 4 disk requests
16 pages read in reverse order: 16 disk requests
16 pages read in order, 2 times: 4 disk requests
write of x to page 3: 0 pages written, the disk holds -
sync: 1 page written, the disk holds x
capacity 4, pages 3 2 1 0 3 used, then page 8: cache holds [0 1 3 8]
write of y to page 1, then pages 0 3 8 12 used: cache holds [0 3 8 12], the disk holds y

Writeback and durability

write returns when the data is in the cache. When the data reaches the disk is decided by the kernel’s flusher threads, which wake periodically and write dirty pages. 4 settings under /proc/sys/vm control them, shown here with the values on this machine.

The last setting is a form of the flow control of chapter 1. A program that writes faster than the disk can absorb fills memory with dirty pages, and at 20% the kernel slows it down to the speed of the disk.

A program for which lost data is unacceptable, such as a database, calls fsync after writing and reports success only after fsync has returned. The call is slow compared with write, since it waits for the disk, and this is the main cost of durability. Opening a file with O_DIRECT bypasses the cache altogether: reads and writes go straight to the disk, for programs that keep a cache of their own.

Eviction

When memory runs short, the kernel frees frames, and the page cache is the first place it looks. It keeps the pages on 2 lists.

A page enters the inactive list when it is first read. If it is used again, it is promoted to the active list. Pages are evicted from the end of the inactive list, and the active list is trimmed into the inactive list to keep the 2 in proportion. The effect is that a page has to be used at least 2 times to be protected, so reading a large file once, as a backup does, does not push out the pages that programs use all the time. /proc/meminfo shows the sizes of the lists as Active(file) and Inactive(file): 3.4 GB and 2.5 GB on this machine.

A clean page is evicted by dropping it. A dirty page has to be written first, which is the reason the kernel writes dirty pages in the background before memory is short. The model above uses a single list with exact access times, which shows the idea with less code.

Inspecting the cache on a live system

The complete programs

Listing 8.19 The complete cachefacts, in Go. code/go/cachefacts/main.go

// Command cachefacts measures the page cache for one file: how many of the
// file's pages are held in memory after a write, after the cache is told to
// drop them, and after reads of different kinds.
//
// Run it from code/go with:
//
//	go run ./cachefacts
package main

import (
	"fmt"
	"io"
	"log"
	"os"
	"os/exec"
	"path/filepath"
	"strconv"
	"strings"

	"golang.org/x/sys/unix"
)


const pageSize = 4096

// cached returns the number of pages of a file that are in the page cache,
// as reported by the fincore utility.
func cached(path string) (int, error) {
	text, err := exec.Command("fincore", "--noheadings", "--output", "PAGES", path).Output()

	if err != nil {
		return 0, err
	}

	return strconv.Atoi(strings.TrimSpace(string(text)))
}

// report prints a step and the number of cached pages after it.
func report(step, path string) error {
	pages, err := cached(path)

	if err != nil {
		return err
	}

	fmt.Printf("%-38s cached pages: %d\n", step+":", pages)

	return nil
}

// drop writes the file's dirty pages to disk and asks the kernel to remove
// the file's pages from the cache.
func drop(file *os.File) error {
	if err := file.Sync(); err != nil {
		return err
	}

	return unix.Fadvise(int(file.Fd()), 0, 0, unix.FADV_DONTNEED)
}


// run writes a file of 64 pages and reads it in 3 ways, dropping the cached
// pages between the steps.
func run(path string) error {
	file, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE, 0o644)

	if err != nil {
		return err
	}

	defer file.Close()

	if _, err := file.Write(make([]byte, 64*pageSize)); err != nil {
		return err
	}

	if err := report("write of 64 pages", path); err != nil {
		return err
	}

	if err := drop(file); err != nil {
		return err
	}

	if err := report("fsync, then FADV_DONTNEED", path); err != nil {
		return err
	}

	one := make([]byte, 1)

	if _, err := file.ReadAt(one, 0); err != nil {
		return err
	}

	if err := report("read of 1 byte at offset 0", path); err != nil {
		return err
	}

	if _, err := io.Copy(io.Discard, io.NewSectionReader(file, 0, 64*pageSize)); err != nil {
		return err
	}

	if err := report("read of the whole file", path); err != nil {
		return err
	}

	if err := drop(file); err != nil {
		return err
	}

	if err := unix.Fadvise(int(file.Fd()), 0, 0, unix.FADV_RANDOM); err != nil {
		return err
	}

	if _, err := file.ReadAt(one, 0); err != nil {
		return err
	}

	return report("dropped, FADV_RANDOM, read of 1 byte", path)
}


func main() {
	dir, err := os.MkdirTemp("", "cachefacts")

	if err != nil {
		log.Fatal(err)
	}

	err = run(filepath.Join(dir, "file"))
	os.RemoveAll(dir)

	if err != nil {
		log.Fatal(err)
	}
}

Listing 8.20 The complete cachefacts, in Rust. code/rust/cachefacts/src/main.rs

//! Measures the page cache for one file: how many of the file's pages are
//! held in memory after a write, after the cache is told to drop them, and
//! after reads of different kinds.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p cachefacts
//! ```

use std::fs::{self, File, OpenOptions};
use std::io::{self, Write};
use std::os::unix::fs::FileExt;
use std::path::Path;
use std::process::Command;

use nix::fcntl::{PosixFadviseAdvice, posix_fadvise};

const PAGE_SIZE: usize = 4096;

/// Returns the number of pages of a file that are in the page cache, as
/// reported by the fincore utility.
fn cached(path: &Path) -> io::Result<usize> {
    let output = Command::new("fincore")
        .args(["--noheadings", "--output", "PAGES"])
        .arg(path)
        .output()?;

    String::from_utf8_lossy(&output.stdout)
        .trim()
        .parse()
        .map_err(|_| io::ErrorKind::InvalidData.into())
}

/// Prints a step and the number of cached pages after it.
fn report(step: &str, path: &Path) -> io::Result<()> {
    let pages = cached(path)?;
    println!("{:<38} cached pages: {pages}", format!("{step}:"));

    Ok(())
}

/// Writes the file's dirty pages to disk and asks the kernel to remove the
/// file's pages from the cache.
fn drop_pages(file: &File) -> io::Result<()> {
    file.sync_all()?;
    posix_fadvise(file, 0, 0, PosixFadviseAdvice::POSIX_FADV_DONTNEED)?;

    Ok(())
}

/// Writes a file of 64 pages and reads it in 3 ways, dropping the cached
/// pages between the steps.
fn run(path: &Path) -> io::Result<()> {
    let mut file = OpenOptions::new()
        .read(true)
        .write(true)
        .create(true)
        .truncate(true)
        .open(path)?;

    file.write_all(&vec![0u8; 64 * PAGE_SIZE])?;
    report("write of 64 pages", path)?;

    drop_pages(&file)?;
    report("fsync, then FADV_DONTNEED", path)?;

    let mut one = [0u8; 1];

    file.read_exact_at(&mut one, 0)?;
    report("read of 1 byte at offset 0", path)?;

    let mut whole = vec![0u8; 64 * PAGE_SIZE];

    file.read_exact_at(&mut whole, 0)?;
    report("read of the whole file", path)?;

    drop_pages(&file)?;
    posix_fadvise(&file, 0, 0, PosixFadviseAdvice::POSIX_FADV_RANDOM)?;

    file.read_exact_at(&mut one, 0)?;
    report("dropped, FADV_RANDOM, read of 1 byte", path)
}

fn main() -> io::Result<()> {
    let dir = std::env::temp_dir().join(format!("cachefacts-{}", std::process::id()));

    fs::create_dir_all(&dir)?;

    let result = run(&dir.join("file"));

    fs::remove_dir_all(&dir)?;

    result
}

Listing 8.21 The complete pagecache, in Go. code/go/pagecache/main.go

// Command pagecache models the kernel's page cache for one file: pages read
// from a disk are kept in memory, sequential reads fetch pages ahead of the
// reader, writes change only the cached page until a sync, and the page used
// least recently is evicted when the cache is full.
//
// Run it from code/go with:
//
//	go run ./pagecache
package main

import (
	"fmt"
	"slices"
)


// maxWindow is the largest number of pages one disk request reads ahead.
const maxWindow = 8

// A disk holds the pages of a file and counts the requests made to it. The
// contents of a page are reduced to one byte.
type disk struct {
	pages    []byte
	requests int
	written  int
}

// A page is a cached copy of a disk page. dirty is set when the copy has
// been written and the disk has not. used is the time of the last access.
type page struct {
	value byte
	dirty bool
	used  int
}

// A cache maps page numbers to cached pages. next is the page that a
// sequential reader would ask for next, and window is the current number of
// pages to read ahead.
type cache struct {
	disk     *disk
	pages    map[int]*page
	capacity int
	clock    int
	next     int
	window   int
}

func newCache(d *disk, capacity int) *cache {
	return &cache{disk: d, pages: map[int]*page{}, capacity: capacity}
}


// read returns the contents of a page. A page that is cached costs no disk
// request. For a page that is missing, read makes one request for a window
// of pages. The window doubles while the reads are sequential and returns
// to 1 page when they are not.
func (c *cache) read(index int) byte {
	if _, hit := c.pages[index]; !hit {
		if index == c.next {
			c.window = min(2*max(c.window, 1), maxWindow)
		} else {
			c.window = 1
		}

		c.load(index, c.window)
	}

	c.next = index + 1

	return c.touch(index).value
}

// load reads count pages starting at index from the disk in one request.
func (c *cache) load(index, count int) {
	c.disk.requests++

	for i := index; i < min(index+count, len(c.disk.pages)); i++ {
		if _, hit := c.pages[i]; !hit {
			c.insert(i, &page{value: c.disk.pages[i]})
		}
	}
}

// touch records an access to a cached page and returns the page.
func (c *cache) touch(index int) *page {
	c.clock++
	c.pages[index].used = c.clock

	return c.pages[index]
}


// write changes a page in the cache and marks it dirty. It makes no disk
// request.
func (c *cache) write(index int, value byte) {
	if _, hit := c.pages[index]; !hit {
		c.insert(index, &page{})
	}

	p := c.touch(index)
	p.value, p.dirty = value, true
}

// sync writes every dirty page to the disk and returns the number written.
func (c *cache) sync() int {
	before := c.disk.written

	for index, p := range c.pages {
		c.clean(index, p)
	}

	return c.disk.written - before
}

// clean writes one page to the disk if it is dirty.
func (c *cache) clean(index int, p *page) {
	if p.dirty {
		c.disk.pages[index] = p.value
		c.disk.written++
		p.dirty = false
	}
}


// insert adds a page to the cache. When the cache is full it first evicts
// the page whose last access is the oldest. A dirty page is written to the
// disk before it is removed.
func (c *cache) insert(index int, p *page) {
	if len(c.pages) == c.capacity {
		oldest := -1

		for i, candidate := range c.pages {
			if oldest == -1 || candidate.used < c.pages[oldest].used {
				oldest = i
			}
		}

		c.clean(oldest, c.pages[oldest])
		delete(c.pages, oldest)
	}

	c.clock++
	p.used = c.clock
	c.pages[index] = p
}

// held returns the numbers of the cached pages in increasing order.
func (c *cache) held() []int {
	held := make([]int, 0, len(c.pages))

	for index := range c.pages {
		held = append(held, index)
	}

	slices.Sort(held)

	return held
}


// requests reads the given pages through a new cache and returns the number
// of disk requests.
func requests(order []int) int {
	d := &disk{pages: make([]byte, 16)}
	c := newCache(d, 32)

	for _, index := range order {
		c.read(index)
	}

	return d.requests
}

func main() {
	forward := make([]int, 16)

	for i := range forward {
		forward[i] = i
	}

	backward := slices.Clone(forward)
	slices.Reverse(backward)

	fmt.Printf("16 pages read in order: %d disk requests\n", requests(forward))
	fmt.Printf("16 pages read in reverse order: %d disk requests\n", requests(backward))
	fmt.Printf("16 pages read in order, 2 times: %d disk requests\n", requests(append(forward, forward...)))

	d := &disk{pages: []byte("----------------")}
	c := newCache(d, 4)

	c.write(3, 'x')
	fmt.Printf("write of x to page 3: %d pages written, the disk holds %c\n", d.written, d.pages[3])
	fmt.Printf("sync: %d page written, the disk holds %c\n", c.sync(), d.pages[3])

	for _, index := range []int{2, 1, 0, 3} {
		c.read(index)
	}

	c.read(8)
	fmt.Printf("capacity 4, pages 3 2 1 0 3 used, then page 8: cache holds %v\n", c.held())

	c.write(1, 'y')

	for _, index := range []int{0, 3, 8, 12} {
		c.read(index)
	}

	fmt.Printf("write of y to page 1, then pages 0 3 8 12 used: cache holds %v, the disk holds %c\n", c.held(), d.pages[1])
}

Listing 8.22 The complete pagecache, in Rust. code/rust/pagecache/src/main.rs

//! Models the kernel's page cache for one file: pages read from a disk are
//! kept in memory, sequential reads fetch pages ahead of the reader, writes
//! change only the cached page until a sync, and the page used least recently
//! is evicted when the cache is full.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p pagecache
//! ```

use std::collections::BTreeMap;

/// The largest number of pages one disk request reads ahead.
const MAX_WINDOW: usize = 8;

/// Holds the pages of a file and counts the requests made to it. The contents
/// of a page are reduced to one byte.
struct Disk {
    pages: Vec<u8>,
    requests: usize,
    written: usize,
}

impl Disk {
    fn new() -> Disk {
        Disk {
            pages: vec![b'-'; 16],
            requests: 0,
            written: 0,
        }
    }
}

/// A cached copy of a disk page. `dirty` is set when the copy has been
/// written and the disk has not. `used` is the time of the last access.
#[derive(Default)]
struct Page {
    value: u8,
    dirty: bool,
    used: usize,
}

/// Maps page numbers to cached pages. `next` is the page that a sequential
/// reader would ask for next, and `window` is the current number of pages to
/// read ahead.
struct Cache {
    disk: Disk,
    pages: BTreeMap<usize, Page>,
    capacity: usize,
    clock: usize,
    next: usize,
    window: usize,
}

impl Cache {
    fn new(capacity: usize) -> Cache {
        Cache {
            disk: Disk::new(),
            pages: BTreeMap::new(),
            capacity,
            clock: 0,
            next: 0,
            window: 0,
        }
    }
}

impl Cache {
    /// Returns the contents of a page. A page that is cached costs no disk
    /// request. For a page that is missing, `read` makes one request for a
    /// window of pages. The window doubles while the reads are sequential and
    /// returns to 1 page when they are not.
    fn read(&mut self, index: usize) -> u8 {
        if !self.pages.contains_key(&index) {
            if index == self.next {
                self.window = (2 * self.window.max(1)).min(MAX_WINDOW);
            } else {
                self.window = 1;
            }

            self.load(index, self.window);
        }

        self.next = index + 1;

        self.touch(index).value
    }

    /// Reads `count` pages starting at `index` from the disk in one request.
    fn load(&mut self, index: usize, count: usize) {
        self.disk.requests += 1;

        for i in index..(index + count).min(self.disk.pages.len()) {
            if !self.pages.contains_key(&i) {
                let value = self.disk.pages[i];

                self.insert(i, value);
            }
        }
    }

    /// Records an access to a page and returns the page. A page that is not
    /// cached is created empty.
    fn touch(&mut self, index: usize) -> &mut Page {
        self.clock += 1;

        let page = self.pages.entry(index).or_default();
        page.used = self.clock;

        page
    }
}

impl Cache {
    /// Changes a page in the cache and marks it dirty. It makes no disk
    /// request.
    fn write(&mut self, index: usize, value: u8) {
        if !self.pages.contains_key(&index) {
            self.insert(index, 0);
        }

        let page = self.touch(index);
        page.value = value;
        page.dirty = true;
    }

    /// Writes every dirty page to the disk and returns the number written.
    fn sync(&mut self) -> usize {
        let before = self.disk.written;

        for (&index, page) in &mut self.pages {
            clean(&mut self.disk, index, page);
        }

        self.disk.written - before
    }
}

/// Writes one page to the disk if it is dirty.
fn clean(disk: &mut Disk, index: usize, page: &mut Page) {
    if page.dirty {
        disk.pages[index] = page.value;
        disk.written += 1;
        page.dirty = false;
    }
}

impl Cache {
    /// Adds a page to the cache. When the cache is full it first evicts the
    /// page whose last access is the oldest. A dirty page is written to the
    /// disk before it is removed.
    fn insert(&mut self, index: usize, value: u8) {
        if self.pages.len() == self.capacity {
            let oldest = self
                .pages
                .iter()
                .min_by_key(|(_, page)| page.used)
                .map(|(&index, _)| index);

            if let Some(oldest) = oldest
                && let Some(mut page) = self.pages.remove(&oldest)
            {
                clean(&mut self.disk, oldest, &mut page);
            }
        }

        self.clock += 1;

        let page = Page {
            value,
            dirty: false,
            used: self.clock,
        };

        self.pages.insert(index, page);
    }

    /// Returns the numbers of the cached pages in increasing order.
    fn held(&self) -> Vec<usize> {
        self.pages.keys().copied().collect()
    }
}

/// Reads the given pages through a new cache and returns the number of disk
/// requests.
fn requests(order: impl Iterator<Item = usize>) -> usize {
    let mut cache = Cache::new(32);

    for index in order {
        cache.read(index);
    }

    cache.disk.requests
}

/// Formats page numbers the way Go prints a slice.
fn list(pages: &[usize]) -> String {
    let words: Vec<String> = pages.iter().map(usize::to_string).collect();

    format!("[{}]", words.join(" "))
}

fn main() {
    println!("16 pages read in order: {} disk requests", requests(0..16));
    println!(
        "16 pages read in reverse order: {} disk requests",
        requests((0..16).rev())
    );
    println!(
        "16 pages read in order, 2 times: {} disk requests",
        requests((0..16).chain(0..16))
    );

    let mut cache = Cache::new(4);

    cache.write(3, b'x');
    println!(
        "write of x to page 3: {} pages written, the disk holds {}",
        cache.disk.written,
        char::from(cache.disk.pages[3])
    );

    let written = cache.sync();
    println!(
        "sync: {written} page written, the disk holds {}",
        char::from(cache.disk.pages[3])
    );

    for index in [2, 1, 0, 3] {
        cache.read(index);
    }

    cache.read(8);
    println!(
        "capacity 4, pages 3 2 1 0 3 used, then page 8: cache holds {}",
        list(&cache.held())
    );

    cache.write(1, b'y');

    for index in [0, 3, 8, 12] {
        cache.read(index);
    }

    println!(
        "write of y to page 1, then pages 0 3 8 12 used: cache holds {}, the disk holds {}",
        list(&cache.held()),
        char::from(cache.disk.pages[1])
    );
}

References

• posix_fadvise(2), fsync(2) and fincore(1), Linux manual pages: https://man7.org/linux/man-pages/man2/posix_fadvise.2.html
• R. Love, Linux Kernel Development, 3rd edition, Addison-Wesley, 2010, chapter 16.
• mm/filemap.c, mm/readahead.c and fs/fs-writeback.c, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/mm/readahead.c

Event Loops and epoll

An event loop is a program structure in which a single thread waits for activity on many file descriptors at once and handles each descriptor when it is ready. The kernel mechanism behind it on Linux is epoll: the program registers descriptors with an epoll instance and then asks which of them are ready. Network servers, the Go runtime and Rust’s async runtimes are all built on this mechanism.

In this chapter, we will learn how epoll finds the ready descriptors without examining all of them. We will first measure its behaviour on pipes with epollfacts. Then we will write eventloop, a model of the interest list and the ready list, check it against the measurements, and run an event loop on it.

Blocking and readiness

By default a read on a pipe or a socket blocks: if no data is available, the kernel puts the calling thread to sleep until some arrives (chapter 1). A thread that is asleep in a read on 1 descriptor cannot notice that another descriptor has data.

A descriptor can be switched to non-blocking mode with the flag O_NONBLOCK. A read that would have to wait then returns the error EAGAIN immediately. A program could call read on each of its descriptors in a loop and skip those that return EAGAIN, and it would consume a whole CPU doing nothing. What is needed is a single call that sleeps until any of a set of descriptors is ready, meaning that a read or a write on it would not block. That call is the subject of this chapter.

The problem

A server with 10000 connections has to respond to whichever connection sends data next. A blocking read waits on 1 descriptor. The alternatives each have a cost.

With select and poll, the program passes all its descriptors on each call, and the kernel checks each one. The work per call is proportional to the number of descriptors registered. On a server most connections are idle at any moment, so most of that work finds nothing.

epoll separates registration from waiting. The program registers a descriptor once, with epoll_ctl. From then on the kernel itself records when the descriptor becomes ready. epoll_wait returns the recorded descriptors, and its work is proportional to the number that are ready.

The ready list is filled at the moment of the event. In chapter 1 a write to a pipe woke the readers sleeping on the pipe’s wait queue. Registering a descriptor with epoll adds an entry to that same wait queue, and the entry’s wake-up function puts the registration on the ready list. The writer does this work as part of its write.

The interface

epoll consists of 3 system calls.

A registration and an event are both a struct epoll_event. This is the definition in include/uapi/linux/eventpoll.h.

struct epoll_event {
        __poll_t events;       /* which conditions: EPOLLIN, EPOLLOUT, ... */
        __u64 data;            /* chosen by the program, returned with each event */
};

data is how the program recognises an event. The kernel does not interpret it. A program stores the descriptor number or a pointer to its own record of the connection, and gets it back in epoll_wait.

A server on a single thread uses the calls in the following pattern.

The descriptors are in non-blocking mode. epoll reports that a descriptor is ready, and the non-blocking mode guarantees that the handler does not sleep if the situation has changed by the time it reads.

From arriving data to epoll_wait

The claim that epoll_wait does work proportional to the number of ready descriptors rests on 1 mechanism, which we follow for a registered descriptor 7.

1. At registration, epoll_ctl creates an item for descriptor 7 and adds an entry to the wait queue of the file behind it. A wait queue is the list of sleepers that the file wakes when its state changes (chapter 1). The entry that epoll adds is a function in place of a sleeping thread.
2. Later, data arrives for descriptor 7. The code that delivers the data, a write by another process or the network stack, wakes the wait queue of the file.
3. Waking the queue calls the function, ep_poll_callback. It puts the item for descriptor 7 on the ready list of the epoll instance.
4. It then wakes any thread that is asleep in epoll_wait on this instance.
5. epoll_wait takes the items from the ready list, asks each file once more whether it is really ready, and copies the events to the program.

No step searches the registered descriptors. The descriptor announces itself at the moment it becomes ready.

The kernel’s structures

An epoll instance is a struct eventpoll, and each registration is a struct epitem. Both are defined in fs/eventpoll.c of Linux 5.15. These are the fields we use.

struct eventpoll {
        struct mutex mtx;                      /* held by epoll_ctl and epoll_wait */
        wait_queue_head_t wq;                  /* threads asleep in epoll_wait */
        struct list_head rdllist;              /* the ready list */
        struct rb_root_cached rbr;             /* the interest list: a tree of epitems */
        /* ... */
};

struct epitem {
        struct rb_node rbn;                    /* its node in rbr */
        struct list_head rdllink;              /* its link in rdllist */
        struct epoll_filefd ffd;               /* the file and descriptor it watches */
        struct eppoll_entry *pwqlist;          /* its entry on the file's wait queue */
        struct eventpoll *ep;                  /* the instance it belongs to */
        struct epoll_event event;              /* the events asked for, and data */
        /* ... */
};

An epoll instance is itself a file, and the kernel shows its registrations in /proc. This is the session manager of our user account, pid 1203, whose descriptor 4 is an epoll instance:

$ ls -l /proc/1203/fd | grep eventpoll
4 -> anon_inode:[eventpoll]
$ cat /proc/1203/fdinfo/4
pos:	0
flags:	02000002
tfd:       29 events:       19 data:     aaaaf18e8be0  pos:0 ino:b06c sdev:8
tfd:       35 events:       19 data:     aaaaf1953a90  pos:0 ino:b06e sdev:8
tfd:       15 events:       19 data:     aaaaf173f750  pos:0 ino:b049 sdev:8

Each tfd line is an item of the interest list: the descriptor it watches, the events in hexadecimal, and data. 19 is EPOLLIN (1), EPOLLERR (8) and EPOLLHUP (0x10) together. The data values are addresses in the memory of the program, which stored a pointer to its own record of each descriptor.

Measuring epoll

epollfacts uses golang.org/x/sys/unix in Go and the nix crate with the event and fs features in Rust.

$ cargo new epollfacts
$ cargo add -p epollfacts nix --features event,fs

watch creates an epoll instance and a number of pipes, and registers the read end of each pipe for the event EPOLLIN, readability. A registration carries a value chosen by the program, here the number of the pipe, which the kernel returns with each event. ready calls epoll_wait with a timeout of 0, so that it reports the current state without blocking.

Listing 9.1 The imports of epollfacts, in Go. code/go/epollfacts/main.go

import (
	"fmt"
	"log"

	"golang.org/x/sys/unix"
)

Listing 9.2 The imports of epollfacts, in Rust. code/rust/epollfacts/src/main.rs

use std::os::fd::OwnedFd;

use nix::fcntl::OFlag;
use nix::sys::epoll::{Epoll, EpollCreateFlags, EpollEvent, EpollFlags, EpollTimeout};
use nix::unistd::{pipe2, read, write};

Listing 9.3 Creating an epoll instance and registering pipes, in Go. code/go/epollfacts/main.go

// A pipe is the pair of descriptors of one pipe.
type pipe struct {
	reader, writer int
}

// watch creates an epoll instance and n pipes, and registers the read end of
// every pipe for readability with the given extra flags. The number of the
// pipe is stored in the registration and comes back with each event.
func watch(n int, flags uint32) (int, []pipe, error) {
	epoll, err := unix.EpollCreate1(unix.EPOLL_CLOEXEC)

	if err != nil {
		return 0, nil, err
	}

	pipes := make([]pipe, n)

	for i := range pipes {
		var fds [2]int

		if err := unix.Pipe2(fds[:], unix.O_CLOEXEC); err != nil {
			return 0, nil, err
		}

		pipes[i] = pipe{reader: fds[0], writer: fds[1]}
		event := unix.EpollEvent{Events: unix.EPOLLIN | flags, Fd: int32(i)}

		if err := unix.EpollCtl(epoll, unix.EPOLL_CTL_ADD, fds[0], &event); err != nil {
			return 0, nil, err
		}
	}

	return epoll, pipes, nil
}

// ready returns the events that are ready now. A timeout of 0 makes the
// wait return at once.
func ready(epoll int) ([]unix.EpollEvent, error) {
	events := make([]unix.EpollEvent, 16)
	n, err := unix.EpollWait(epoll, events, 0)

	if err != nil {
		return nil, err
	}

	return events[:n], nil
}

// release closes an epoll instance and its pipes.
func release(epoll int, pipes []pipe) {
	unix.Close(epoll)

	for _, p := range pipes {
		unix.Close(p.reader)
		unix.Close(p.writer)
	}
}

Listing 9.4 Creating an epoll instance and registering pipes, in Rust. code/rust/epollfacts/src/main.rs

/// The pair of descriptors of one pipe. The write end is an `Option` so that
/// an experiment can close it.
struct Pipe {
    reader: OwnedFd,
    writer: Option<OwnedFd>,
}

/// Creates an epoll instance and `n` pipes, and registers the read end of
/// every pipe for readability with the given extra flags. The number of the
/// pipe is stored in the registration and comes back with each event.
fn watch(n: usize, flags: EpollFlags) -> nix::Result<(Epoll, Vec<Pipe>)> {
    let epoll = Epoll::new(EpollCreateFlags::EPOLL_CLOEXEC)?;
    let mut pipes = Vec::with_capacity(n);

    for i in 0..n {
        let (reader, writer) = pipe2(OFlag::O_CLOEXEC)?;

        epoll.add(
            &reader,
            EpollEvent::new(EpollFlags::EPOLLIN | flags, i as u64),
        )?;

        pipes.push(Pipe {
            reader,
            writer: Some(writer),
        });
    }

    Ok((epoll, pipes))
}

/// Returns the events that are ready now. A timeout of 0 makes the wait
/// return at once.
fn ready(epoll: &Epoll) -> nix::Result<Vec<EpollEvent>> {
    let mut events = [EpollEvent::empty(); 16];
    let n = epoll.wait(&mut events, EpollTimeout::ZERO)?;

    Ok(events[..n].to_vec())
}

/// Writes "hello" to a pipe whose write end is open.
fn send(pipe: &Pipe) -> nix::Result<()> {
    if let Some(writer) = &pipe.writer {
        write(writer, b"hello")?;
    }

    Ok(())
}

level uses the default kind of registration, called level-triggered. It waits before any write, after a write to pipe 1, a second time without reading the data, after reading it, and after closing the write end.

Listing 9.5 Level-triggered events, in Go. code/go/epollfacts/main.go

// level registers 3 pipes level-triggered and waits before a write, after a
// write, again without reading, and after reading.
func level() error {
	epoll, pipes, err := watch(3, 0)

	if err != nil {
		return err
	}

	defer release(epoll, pipes)

	events, err := ready(epoll)

	if err != nil {
		return err
	}

	fmt.Printf("3 pipes registered, none written: %d event(s)\n", len(events))

	if _, err := unix.Write(pipes[1].writer, []byte("hello")); err != nil {
		return err
	}

	events, err = ready(epoll)

	if err != nil || len(events) != 1 {
		return fmt.Errorf("wait after a write: %d events, %v", len(events), err)
	}

	fmt.Printf("write to pipe 1: 1 event, for pipe %d\n", events[0].Fd)

	events, err = ready(epoll)

	if err != nil {
		return err
	}

	fmt.Printf("level-triggered, data not read, wait again: %d event(s)\n", len(events))

	if _, err := unix.Read(pipes[1].reader, make([]byte, 16)); err != nil {
		return err
	}

	events, err = ready(epoll)

	if err != nil {
		return err
	}

	fmt.Printf("after reading the data: %d event(s)\n", len(events))

	unix.Close(pipes[1].writer)
	events, err = ready(epoll)

	if err != nil || len(events) != 1 {
		return fmt.Errorf("wait after a close: %d events, %v", len(events), err)
	}

	fmt.Printf("write end of pipe 1 closed: event has EPOLLHUP %v\n", events[0].Events&unix.EPOLLHUP != 0)

	return nil
}

Listing 9.6 Level-triggered events, in Rust. code/rust/epollfacts/src/main.rs

/// Registers 3 pipes level-triggered and waits before a write, after a write,
/// again without reading, and after reading.
fn level() -> nix::Result<()> {
    let (epoll, mut pipes) = watch(3, EpollFlags::empty())?;

    println!(
        "3 pipes registered, none written: {} event(s)",
        ready(&epoll)?.len()
    );

    send(&pipes[1])?;

    if let [event] = ready(&epoll)?.as_slice() {
        println!("write to pipe 1: 1 event, for pipe {}", event.data());
    }

    println!(
        "level-triggered, data not read, wait again: {} event(s)",
        ready(&epoll)?.len()
    );

    read(&pipes[1].reader, &mut [0u8; 16])?;
    println!("after reading the data: {} event(s)", ready(&epoll)?.len());

    pipes[1].writer = None;

    if let [event] = ready(&epoll)?.as_slice() {
        println!(
            "write end of pipe 1 closed: event has EPOLLHUP {}",
            event.events().contains(EpollFlags::EPOLLHUP)
        );
    }

    Ok(())
}

edge registers a pipe with the flag EPOLLET, edge-triggered, and waits 2 times after a write and again after a further write.

Listing 9.7 Edge-triggered events, in Go. code/go/epollfacts/main.go

// edge registers a pipe edge-triggered and waits 2 times after one write,
// and once more after a second write.
func edge() error {
	epoll, pipes, err := watch(1, unix.EPOLLET)

	if err != nil {
		return err
	}

	defer release(epoll, pipes)

	var counts [3]int

	for i := range counts {
		if i != 1 {
			if _, err := unix.Write(pipes[0].writer, []byte("hello")); err != nil {
				return err
			}
		}

		events, err := ready(epoll)

		if err != nil {
			return err
		}

		counts[i] = len(events)
	}

	fmt.Printf("edge-triggered, data not read: wait %d event(s), wait again %d event(s)\n", counts[0], counts[1])
	fmt.Printf("edge-triggered, after another write: %d event(s)\n", counts[2])

	return nil
}

Listing 9.8 Edge-triggered events, in Rust. code/rust/epollfacts/src/main.rs

/// Registers a pipe edge-triggered and waits 2 times after one write, and
/// once more after a second write.
fn edge() -> nix::Result<()> {
    let (epoll, pipes) = watch(1, EpollFlags::EPOLLET)?;
    let mut counts = [0; 3];

    for (i, count) in counts.iter_mut().enumerate() {
        if i != 1 {
            send(&pipes[0])?;
        }

        *count = ready(&epoll)?.len();
    }

    println!(
        "edge-triggered, data not read: wait {} event(s), wait again {} event(s)",
        counts[0], counts[1]
    );
    println!(
        "edge-triggered, after another write: {} event(s)",
        counts[2]
    );

    Ok(())
}

many registers 1000 pipes and writes to pipe 700.

Listing 9.9 1000 registrations and 1 ready descriptor, in Go. code/go/epollfacts/main.go

// many registers 1000 pipes and writes to 1 of them.
func many() error {
	epoll, pipes, err := watch(1000, 0)

	if err != nil {
		return err
	}

	defer release(epoll, pipes)

	if _, err := unix.Write(pipes[700].writer, []byte("hello")); err != nil {
		return err
	}

	events, err := ready(epoll)

	if err != nil || len(events) != 1 {
		return fmt.Errorf("wait with 1000 pipes: %d events, %v", len(events), err)
	}

	fmt.Printf("1000 pipes registered, pipe 700 written: 1 event, for pipe %d\n", events[0].Fd)

	return nil
}

Listing 9.10 1000 registrations and 1 ready descriptor, in Rust. code/rust/epollfacts/src/main.rs

/// Registers 1000 pipes and writes to 1 of them.
fn many() -> nix::Result<()> {
    let (epoll, pipes) = watch(1000, EpollFlags::empty())?;

    send(&pipes[700])?;

    if let [event] = ready(&epoll)?.as_slice() {
        println!(
            "1000 pipes registered, pipe 700 written: 1 event, for pipe {}",
            event.data()
        );
    }

    Ok(())
}

3 pipes registered, none written: 0 event(s)
write to pipe 1: 1 event, for pipe 1
level-triggered, data not read, wait again: 1 event(s)
after reading the data: 0 event(s)
write end of pipe 1 closed: event has EPOLLHUP true
edge-triggered, data not read: wait 1 event(s), wait again 0 event(s)
edge-triggered, after another write: 1 event(s)
1000 pipes registered, pipe 700 written: 1 event, for pipe 700

The choice has a consequence for the handler. With a level-triggered registration, a handler may read part of the data and return; the descriptor is reported again. With an edge-triggered registration, a handler has to read until read returns EAGAIN, because no further event will announce the data it leaves behind. Edge-triggered registrations save repeated reports and are used by runtimes that follow this rule strictly.

Modelling epoll

A pipe is reduced to a count of buffered bytes. The queued flag keeps a pipe from appearing on the ready list more than once.

Listing 9.11 The imports of eventloop, in Go. code/go/eventloop/main.go

import "fmt"

Listing 9.12 Pipes, the interest list and the ready list, in Go. code/go/eventloop/main.go

// A pipe is the read end of a pipe as epoll sees it: the bytes waiting to be
// read, and its registration. queued is set while the pipe is on the ready
// list.
type pipe struct {
	buffered   int
	registered bool
	edge       bool
	queued     bool
}

// A kernel holds the pipes and one epoll instance. The registered pipes are
// the interest list. ready is the ready list, in the order in which the
// pipes became ready. examined counts the registrations that waits and scans
// have looked at.
type kernel struct {
	pipes    []pipe
	ready    []int
	examined int
}

func newKernel(pipes int) *kernel {
	return &kernel{pipes: make([]pipe, pipes)}
}

Listing 9.13 Pipes, the interest list and the ready list, in Rust. code/rust/eventloop/src/main.rs

/// The read end of a pipe as epoll sees it: the bytes waiting to be read, and
/// its registration. `queued` is set while the pipe is on the ready list.
#[derive(Clone, Default)]
struct Pipe {
    buffered: usize,
    registered: bool,
    edge: bool,
    queued: bool,
}

/// Holds the pipes and one epoll instance. The registered pipes are the
/// interest list. `ready` is the ready list, in the order in which the pipes
/// became ready. `examined` counts the registrations that waits and scans
/// have looked at.
struct Kernel {
    pipes: Vec<Pipe>,
    ready: Vec<usize>,
    examined: usize,
}

impl Kernel {
    fn new(pipes: usize) -> Kernel {
        Kernel {
            pipes: vec![Pipe::default(); pipes],
            ready: Vec::new(),
            examined: 0,
        }
    }
}

write is where a pipe becomes ready, and it is write that queues the pipe. No part of the model searches for ready pipes.

Listing 9.14 Registering, and queuing on a write, in Go. code/go/eventloop/main.go

// register adds a pipe to the interest list. A pipe that is already
// readable goes on the ready list at once.
func (k *kernel) register(index int, edge bool) {
	p := &k.pipes[index]
	p.registered, p.edge = true, edge

	if p.buffered > 0 {
		k.enqueue(index)
	}
}

// enqueue puts a pipe on the ready list unless it is already there.
func (k *kernel) enqueue(index int) {
	if p := &k.pipes[index]; !p.queued {
		p.queued = true
		k.ready = append(k.ready, index)
	}
}

// write adds bytes to a pipe. This is the moment the pipe becomes ready, and
// the writer puts it on the ready list.
func (k *kernel) write(index, n int) {
	k.pipes[index].buffered += n

	if k.pipes[index].registered {
		k.enqueue(index)
	}
}

// read removes up to n bytes from a pipe and returns the number removed.
func (k *kernel) read(index, n int) int {
	n = min(n, k.pipes[index].buffered)
	k.pipes[index].buffered -= n

	return n
}

Listing 9.15 Registering, and queuing on a write, in Rust. code/rust/eventloop/src/main.rs

impl Kernel {
    /// Adds a pipe to the interest list. A pipe that is already readable goes
    /// on the ready list at once.
    fn register(&mut self, index: usize, edge: bool) {
        let pipe = &mut self.pipes[index];

        pipe.registered = true;
        pipe.edge = edge;

        if pipe.buffered > 0 {
            self.enqueue(index);
        }
    }

    /// Puts a pipe on the ready list unless it is already there.
    fn enqueue(&mut self, index: usize) {
        if !self.pipes[index].queued {
            self.pipes[index].queued = true;
            self.ready.push(index);
        }
    }

    /// Adds bytes to a pipe. This is the moment the pipe becomes ready, and
    /// the writer puts it on the ready list.
    fn write(&mut self, index: usize, n: usize) {
        self.pipes[index].buffered += n;

        if self.pipes[index].registered {
            self.enqueue(index);
        }
    }

    /// Removes up to `n` bytes from a pipe and returns the number removed.
    fn read(&mut self, index: usize, n: usize) -> usize {
        let n = n.min(self.pipes[index].buffered);
        self.pipes[index].buffered -= n;

        n
    }
}

wait takes the ready list and checks each entry again, since the data may have been read in the meantime. The difference between the kinds of registration is a single statement: a level-triggered pipe that is reported is queued again, and an edge-triggered pipe is left off the list until the next write queues it.

Listing 9.16 Waiting, and scanning for comparison, in Go. code/go/eventloop/main.go

// wait empties the ready list and returns the pipes on it that are still
// readable. A level-triggered pipe that is reported goes back on the list,
// so the next wait checks it again. An edge-triggered pipe does not, and is
// reported again only after another write.
func (k *kernel) wait() []int {
	queued := k.ready
	k.ready = nil

	var events []int

	for _, index := range queued {
		p := &k.pipes[index]
		p.queued = false
		k.examined++

		if p.buffered == 0 {
			continue
		}

		events = append(events, index)

		if !p.edge {
			k.enqueue(index)
		}
	}

	return events
}

// scan is the method of select and poll: it examines every registered pipe
// on every call.
func (k *kernel) scan() []int {
	var events []int

	for index, p := range k.pipes {
		if !p.registered {
			continue
		}

		k.examined++

		if p.buffered > 0 {
			events = append(events, index)
		}
	}

	return events
}

Listing 9.17 Waiting, and scanning for comparison, in Rust. code/rust/eventloop/src/main.rs

impl Kernel {
    /// Empties the ready list and returns the pipes on it that are still
    /// readable. A level-triggered pipe that is reported goes back on the
    /// list, so the next wait checks it again. An edge-triggered pipe does
    /// not, and is reported again only after another write.
    fn wait(&mut self) -> Vec<usize> {
        let queued = std::mem::take(&mut self.ready);
        let mut events = Vec::new();

        for index in queued {
            self.pipes[index].queued = false;
            self.examined += 1;

            if self.pipes[index].buffered == 0 {
                continue;
            }

            events.push(index);

            if !self.pipes[index].edge {
                self.enqueue(index);
            }
        }

        events
    }

    /// The method of select and poll: it examines every registered pipe on
    /// every call.
    fn scan(&mut self) -> Vec<usize> {
        let mut events = Vec::new();

        for (index, pipe) in self.pipes.iter().enumerate() {
            if !pipe.registered {
                continue;
            }

            self.examined += 1;

            if pipe.buffered > 0 {
                events.push(index);
            }
        }

        events
    }
}

The model repeats the measurements. many also counts how many registrations a wait and a scan examine.

Listing 9.18 The measurements on the model, in Go. code/go/eventloop/main.go

// level repeats the level-triggered measurement of epollfacts.
func level() {
	k := newKernel(3)

	for index := range k.pipes {
		k.register(index, false)
	}

	fmt.Printf("3 pipes registered, none written: %d event(s)\n", len(k.wait()))

	k.write(1, 5)

	if events := k.wait(); len(events) == 1 {
		fmt.Printf("write to pipe 1: 1 event, for pipe %d\n", events[0])
	}

	fmt.Printf("level-triggered, data not read, wait again: %d event(s)\n", len(k.wait()))

	k.read(1, 16)
	fmt.Printf("after reading the data: %d event(s)\n", len(k.wait()))
}

// edge repeats the edge-triggered measurement.
func edge() {
	k := newKernel(1)
	k.register(0, true)

	k.write(0, 5)
	first, second := len(k.wait()), len(k.wait())
	fmt.Printf("edge-triggered, data not read: wait %d event(s), wait again %d event(s)\n", first, second)

	k.write(0, 5)
	fmt.Printf("edge-triggered, after another write: %d event(s)\n", len(k.wait()))
}

// many compares the work of a wait with the work of a scan when 1000 pipes
// are registered and 1 is ready.
func many() {
	k := newKernel(1000)

	for index := range k.pipes {
		k.register(index, false)
	}

	k.write(700, 5)

	if events := k.wait(); len(events) == 1 {
		fmt.Printf("1000 pipes registered, pipe 700 written: 1 event, for pipe %d\n", events[0])
	}

	byWait := k.examined
	k.scan()
	fmt.Printf("registrations examined: %d by the wait, %d by a scan\n", byWait, k.examined-byWait)
}

Listing 9.19 The measurements on the model, in Rust. code/rust/eventloop/src/main.rs

/// Repeats the level-triggered measurement of epollfacts.
fn level() {
    let mut kernel = Kernel::new(3);

    for index in 0..3 {
        kernel.register(index, false);
    }

    println!(
        "3 pipes registered, none written: {} event(s)",
        kernel.wait().len()
    );

    kernel.write(1, 5);

    if let [index] = kernel.wait().as_slice() {
        println!("write to pipe 1: 1 event, for pipe {index}");
    }

    println!(
        "level-triggered, data not read, wait again: {} event(s)",
        kernel.wait().len()
    );

    kernel.read(1, 16);
    println!("after reading the data: {} event(s)", kernel.wait().len());
}

/// Repeats the edge-triggered measurement.
fn edge() {
    let mut kernel = Kernel::new(1);
    kernel.register(0, true);

    kernel.write(0, 5);
    let (first, second) = (kernel.wait().len(), kernel.wait().len());
    println!("edge-triggered, data not read: wait {first} event(s), wait again {second} event(s)");

    kernel.write(0, 5);
    println!(
        "edge-triggered, after another write: {} event(s)",
        kernel.wait().len()
    );
}

/// Compares the work of a wait with the work of a scan when 1000 pipes are
/// registered and 1 is ready.
fn many() {
    let mut kernel = Kernel::new(1000);

    for index in 0..1000 {
        kernel.register(index, false);
    }

    kernel.write(700, 5);

    if let [index] = kernel.wait().as_slice() {
        println!("1000 pipes registered, pipe 700 written: 1 event, for pipe {index}");
    }

    let by_wait = kernel.examined;
    kernel.scan();
    println!(
        "registrations examined: {by_wait} by the wait, {} by a scan",
        kernel.examined - by_wait
    );
}

The loop

An event loop repeats 2 steps: wait for events, then run the handler of each ready descriptor. A handler must return quickly and must not block, because every other descriptor waits while it runs.

Listing 9.20 An event loop on the model, in Go. code/go/eventloop/main.go

// turn is one iteration of an event loop: wait for ready pipes, then run the
// handler for each. It returns the pipes it handled.
func (k *kernel) turn(handle func(k *kernel, index int)) []int {
	events := k.wait()

	for _, index := range events {
		handle(k, index)
	}

	return events
}

// loop registers 4 pipes, writes to 2 of them, and runs an event loop whose
// handler reads at most 4 bytes for each event.
func loop() {
	k := newKernel(4)

	for index := range k.pipes {
		k.register(index, false)
	}

	k.write(2, 5)
	k.write(0, 3)

	handle := func(k *kernel, index int) { k.read(index, 4) }

	fmt.Printf("event loop: turn 1 handles %v, turn 2 handles %v, turn 3 handles %v\n",
		k.turn(handle), k.turn(handle), k.turn(handle))
}

Listing 9.21 An event loop on the model, in Rust. code/rust/eventloop/src/main.rs

impl Kernel {
    /// One iteration of an event loop: wait for ready pipes, then run the
    /// handler for each. Returns the pipes it handled.
    fn turn(&mut self, handle: fn(&mut Kernel, usize)) -> Vec<usize> {
        let events = self.wait();

        for &index in &events {
            handle(self, index);
        }

        events
    }
}

/// Formats pipe numbers the way Go prints a slice.
fn list(pipes: &[usize]) -> String {
    let words: Vec<String> = pipes.iter().map(usize::to_string).collect();

    format!("[{}]", words.join(" "))
}

/// Registers 4 pipes, writes to 2 of them, and runs an event loop whose
/// handler reads at most 4 bytes for each event.
fn event_loop() {
    let mut kernel = Kernel::new(4);

    for index in 0..4 {
        kernel.register(index, false);
    }

    kernel.write(2, 5);
    kernel.write(0, 3);

    let handle = |kernel: &mut Kernel, index: usize| {
        kernel.read(index, 4);
    };

    let turns = [
        kernel.turn(handle),
        kernel.turn(handle),
        kernel.turn(handle),
    ];

    println!(
        "event loop: turn 1 handles {}, turn 2 handles {}, turn 3 handles {}",
        list(&turns[0]),
        list(&turns[1]),
        list(&turns[2])
    );
}

3 pipes registered, none written: 0 event(s)
write to pipe 1: 1 event, for pipe 1
level-triggered, data not read, wait again: 1 event(s)
after reading the data: 0 event(s)
edge-triggered, data not read: wait 1 event(s), wait again 0 event(s)
edge-triggered, after another write: 1 event(s)
1000 pipes registered, pipe 700 written: 1 event, for pipe 700
registrations examined: 1 by the wait, 1000 by a scan
event loop: turn 1 handles [2 0], turn 2 handles [2], turn 3 handles []

Epoll inside language runtimes

Go and Rust programs rarely call epoll directly, and nearly all of their network I/O goes through it.

The Go runtime has a component called the netpoller. Every socket a Go program opens is put in non-blocking mode and registered, edge-triggered, with a single epoll instance. When a goroutine calls Read and the kernel answers EAGAIN, the runtime parks the goroutine, which costs no thread, and runs another. A wake-up from epoll makes the parked goroutine runnable again. The code reads as if it blocked, and underneath it is the event loop of this chapter.

In Rust, an async runtime such as Tokio does the same for futures. A future that cannot make progress registers its descriptor and returns Pending. The runtime’s reactor calls epoll_wait, and wakes the task that owns a descriptor when its event arrives.

Inspecting epoll on a live system

The complete programs

Listing 9.22 The complete epollfacts, in Go. code/go/epollfacts/main.go

// Command epollfacts measures epoll with pipes: which descriptors a wait
// reports, how level-triggered and edge-triggered registrations differ, how
// a closed peer is reported, and how many events a wait returns when 1000
// descriptors are registered and 1 is ready.
//
// Run it from code/go with:
//
//	go run ./epollfacts
package main

import (
	"fmt"
	"log"

	"golang.org/x/sys/unix"
)


// A pipe is the pair of descriptors of one pipe.
type pipe struct {
	reader, writer int
}

// watch creates an epoll instance and n pipes, and registers the read end of
// every pipe for readability with the given extra flags. The number of the
// pipe is stored in the registration and comes back with each event.
func watch(n int, flags uint32) (int, []pipe, error) {
	epoll, err := unix.EpollCreate1(unix.EPOLL_CLOEXEC)

	if err != nil {
		return 0, nil, err
	}

	pipes := make([]pipe, n)

	for i := range pipes {
		var fds [2]int

		if err := unix.Pipe2(fds[:], unix.O_CLOEXEC); err != nil {
			return 0, nil, err
		}

		pipes[i] = pipe{reader: fds[0], writer: fds[1]}
		event := unix.EpollEvent{Events: unix.EPOLLIN | flags, Fd: int32(i)}

		if err := unix.EpollCtl(epoll, unix.EPOLL_CTL_ADD, fds[0], &event); err != nil {
			return 0, nil, err
		}
	}

	return epoll, pipes, nil
}

// ready returns the events that are ready now. A timeout of 0 makes the
// wait return at once.
func ready(epoll int) ([]unix.EpollEvent, error) {
	events := make([]unix.EpollEvent, 16)
	n, err := unix.EpollWait(epoll, events, 0)

	if err != nil {
		return nil, err
	}

	return events[:n], nil
}

// release closes an epoll instance and its pipes.
func release(epoll int, pipes []pipe) {
	unix.Close(epoll)

	for _, p := range pipes {
		unix.Close(p.reader)
		unix.Close(p.writer)
	}
}


// level registers 3 pipes level-triggered and waits before a write, after a
// write, again without reading, and after reading.
func level() error {
	epoll, pipes, err := watch(3, 0)

	if err != nil {
		return err
	}

	defer release(epoll, pipes)

	events, err := ready(epoll)

	if err != nil {
		return err
	}

	fmt.Printf("3 pipes registered, none written: %d event(s)\n", len(events))

	if _, err := unix.Write(pipes[1].writer, []byte("hello")); err != nil {
		return err
	}

	events, err = ready(epoll)

	if err != nil || len(events) != 1 {
		return fmt.Errorf("wait after a write: %d events, %v", len(events), err)
	}

	fmt.Printf("write to pipe 1: 1 event, for pipe %d\n", events[0].Fd)

	events, err = ready(epoll)

	if err != nil {
		return err
	}

	fmt.Printf("level-triggered, data not read, wait again: %d event(s)\n", len(events))

	if _, err := unix.Read(pipes[1].reader, make([]byte, 16)); err != nil {
		return err
	}

	events, err = ready(epoll)

	if err != nil {
		return err
	}

	fmt.Printf("after reading the data: %d event(s)\n", len(events))

	unix.Close(pipes[1].writer)
	events, err = ready(epoll)

	if err != nil || len(events) != 1 {
		return fmt.Errorf("wait after a close: %d events, %v", len(events), err)
	}

	fmt.Printf("write end of pipe 1 closed: event has EPOLLHUP %v\n", events[0].Events&unix.EPOLLHUP != 0)

	return nil
}


// edge registers a pipe edge-triggered and waits 2 times after one write,
// and once more after a second write.
func edge() error {
	epoll, pipes, err := watch(1, unix.EPOLLET)

	if err != nil {
		return err
	}

	defer release(epoll, pipes)

	var counts [3]int

	for i := range counts {
		if i != 1 {
			if _, err := unix.Write(pipes[0].writer, []byte("hello")); err != nil {
				return err
			}
		}

		events, err := ready(epoll)

		if err != nil {
			return err
		}

		counts[i] = len(events)
	}

	fmt.Printf("edge-triggered, data not read: wait %d event(s), wait again %d event(s)\n", counts[0], counts[1])
	fmt.Printf("edge-triggered, after another write: %d event(s)\n", counts[2])

	return nil
}


// many registers 1000 pipes and writes to 1 of them.
func many() error {
	epoll, pipes, err := watch(1000, 0)

	if err != nil {
		return err
	}

	defer release(epoll, pipes)

	if _, err := unix.Write(pipes[700].writer, []byte("hello")); err != nil {
		return err
	}

	events, err := ready(epoll)

	if err != nil || len(events) != 1 {
		return fmt.Errorf("wait with 1000 pipes: %d events, %v", len(events), err)
	}

	fmt.Printf("1000 pipes registered, pipe 700 written: 1 event, for pipe %d\n", events[0].Fd)

	return nil
}


func main() {
	for _, experiment := range []func() error{level, edge, many} {
		if err := experiment(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 9.23 The complete epollfacts, in Rust. code/rust/epollfacts/src/main.rs

//! Measures epoll with pipes: which descriptors a wait reports, how
//! level-triggered and edge-triggered registrations differ, how a closed peer
//! is reported, and how many events a wait returns when 1000 descriptors are
//! registered and 1 is ready.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p epollfacts
//! ```

use std::os::fd::OwnedFd;

use nix::fcntl::OFlag;
use nix::sys::epoll::{Epoll, EpollCreateFlags, EpollEvent, EpollFlags, EpollTimeout};
use nix::unistd::{pipe2, read, write};

/// The pair of descriptors of one pipe. The write end is an `Option` so that
/// an experiment can close it.
struct Pipe {
    reader: OwnedFd,
    writer: Option<OwnedFd>,
}

/// Creates an epoll instance and `n` pipes, and registers the read end of
/// every pipe for readability with the given extra flags. The number of the
/// pipe is stored in the registration and comes back with each event.
fn watch(n: usize, flags: EpollFlags) -> nix::Result<(Epoll, Vec<Pipe>)> {
    let epoll = Epoll::new(EpollCreateFlags::EPOLL_CLOEXEC)?;
    let mut pipes = Vec::with_capacity(n);

    for i in 0..n {
        let (reader, writer) = pipe2(OFlag::O_CLOEXEC)?;

        epoll.add(
            &reader,
            EpollEvent::new(EpollFlags::EPOLLIN | flags, i as u64),
        )?;

        pipes.push(Pipe {
            reader,
            writer: Some(writer),
        });
    }

    Ok((epoll, pipes))
}

/// Returns the events that are ready now. A timeout of 0 makes the wait
/// return at once.
fn ready(epoll: &Epoll) -> nix::Result<Vec<EpollEvent>> {
    let mut events = [EpollEvent::empty(); 16];
    let n = epoll.wait(&mut events, EpollTimeout::ZERO)?;

    Ok(events[..n].to_vec())
}

/// Writes "hello" to a pipe whose write end is open.
fn send(pipe: &Pipe) -> nix::Result<()> {
    if let Some(writer) = &pipe.writer {
        write(writer, b"hello")?;
    }

    Ok(())
}

/// Registers 3 pipes level-triggered and waits before a write, after a write,
/// again without reading, and after reading.
fn level() -> nix::Result<()> {
    let (epoll, mut pipes) = watch(3, EpollFlags::empty())?;

    println!(
        "3 pipes registered, none written: {} event(s)",
        ready(&epoll)?.len()
    );

    send(&pipes[1])?;

    if let [event] = ready(&epoll)?.as_slice() {
        println!("write to pipe 1: 1 event, for pipe {}", event.data());
    }

    println!(
        "level-triggered, data not read, wait again: {} event(s)",
        ready(&epoll)?.len()
    );

    read(&pipes[1].reader, &mut [0u8; 16])?;
    println!("after reading the data: {} event(s)", ready(&epoll)?.len());

    pipes[1].writer = None;

    if let [event] = ready(&epoll)?.as_slice() {
        println!(
            "write end of pipe 1 closed: event has EPOLLHUP {}",
            event.events().contains(EpollFlags::EPOLLHUP)
        );
    }

    Ok(())
}

/// Registers a pipe edge-triggered and waits 2 times after one write, and
/// once more after a second write.
fn edge() -> nix::Result<()> {
    let (epoll, pipes) = watch(1, EpollFlags::EPOLLET)?;
    let mut counts = [0; 3];

    for (i, count) in counts.iter_mut().enumerate() {
        if i != 1 {
            send(&pipes[0])?;
        }

        *count = ready(&epoll)?.len();
    }

    println!(
        "edge-triggered, data not read: wait {} event(s), wait again {} event(s)",
        counts[0], counts[1]
    );
    println!(
        "edge-triggered, after another write: {} event(s)",
        counts[2]
    );

    Ok(())
}

/// Registers 1000 pipes and writes to 1 of them.
fn many() -> nix::Result<()> {
    let (epoll, pipes) = watch(1000, EpollFlags::empty())?;

    send(&pipes[700])?;

    if let [event] = ready(&epoll)?.as_slice() {
        println!(
            "1000 pipes registered, pipe 700 written: 1 event, for pipe {}",
            event.data()
        );
    }

    Ok(())
}

fn main() -> nix::Result<()> {
    level()?;
    edge()?;
    many()
}

Listing 9.24 The complete eventloop, in Go. code/go/eventloop/main.go

// Command eventloop models epoll: an interest list of registered pipes, a
// ready list that a write fills, and a wait that examines only the ready
// list. It repeats the measurements of epollfacts on the model and then runs
// a small event loop on it.
//
// Run it from code/go with:
//
//	go run ./eventloop
package main

import "fmt"


// A pipe is the read end of a pipe as epoll sees it: the bytes waiting to be
// read, and its registration. queued is set while the pipe is on the ready
// list.
type pipe struct {
	buffered   int
	registered bool
	edge       bool
	queued     bool
}

// A kernel holds the pipes and one epoll instance. The registered pipes are
// the interest list. ready is the ready list, in the order in which the
// pipes became ready. examined counts the registrations that waits and scans
// have looked at.
type kernel struct {
	pipes    []pipe
	ready    []int
	examined int
}

func newKernel(pipes int) *kernel {
	return &kernel{pipes: make([]pipe, pipes)}
}


// register adds a pipe to the interest list. A pipe that is already
// readable goes on the ready list at once.
func (k *kernel) register(index int, edge bool) {
	p := &k.pipes[index]
	p.registered, p.edge = true, edge

	if p.buffered > 0 {
		k.enqueue(index)
	}
}

// enqueue puts a pipe on the ready list unless it is already there.
func (k *kernel) enqueue(index int) {
	if p := &k.pipes[index]; !p.queued {
		p.queued = true
		k.ready = append(k.ready, index)
	}
}

// write adds bytes to a pipe. This is the moment the pipe becomes ready, and
// the writer puts it on the ready list.
func (k *kernel) write(index, n int) {
	k.pipes[index].buffered += n

	if k.pipes[index].registered {
		k.enqueue(index)
	}
}

// read removes up to n bytes from a pipe and returns the number removed.
func (k *kernel) read(index, n int) int {
	n = min(n, k.pipes[index].buffered)
	k.pipes[index].buffered -= n

	return n
}


// wait empties the ready list and returns the pipes on it that are still
// readable. A level-triggered pipe that is reported goes back on the list,
// so the next wait checks it again. An edge-triggered pipe does not, and is
// reported again only after another write.
func (k *kernel) wait() []int {
	queued := k.ready
	k.ready = nil

	var events []int

	for _, index := range queued {
		p := &k.pipes[index]
		p.queued = false
		k.examined++

		if p.buffered == 0 {
			continue
		}

		events = append(events, index)

		if !p.edge {
			k.enqueue(index)
		}
	}

	return events
}

// scan is the method of select and poll: it examines every registered pipe
// on every call.
func (k *kernel) scan() []int {
	var events []int

	for index, p := range k.pipes {
		if !p.registered {
			continue
		}

		k.examined++

		if p.buffered > 0 {
			events = append(events, index)
		}
	}

	return events
}


// level repeats the level-triggered measurement of epollfacts.
func level() {
	k := newKernel(3)

	for index := range k.pipes {
		k.register(index, false)
	}

	fmt.Printf("3 pipes registered, none written: %d event(s)\n", len(k.wait()))

	k.write(1, 5)

	if events := k.wait(); len(events) == 1 {
		fmt.Printf("write to pipe 1: 1 event, for pipe %d\n", events[0])
	}

	fmt.Printf("level-triggered, data not read, wait again: %d event(s)\n", len(k.wait()))

	k.read(1, 16)
	fmt.Printf("after reading the data: %d event(s)\n", len(k.wait()))
}

// edge repeats the edge-triggered measurement.
func edge() {
	k := newKernel(1)
	k.register(0, true)

	k.write(0, 5)
	first, second := len(k.wait()), len(k.wait())
	fmt.Printf("edge-triggered, data not read: wait %d event(s), wait again %d event(s)\n", first, second)

	k.write(0, 5)
	fmt.Printf("edge-triggered, after another write: %d event(s)\n", len(k.wait()))
}

// many compares the work of a wait with the work of a scan when 1000 pipes
// are registered and 1 is ready.
func many() {
	k := newKernel(1000)

	for index := range k.pipes {
		k.register(index, false)
	}

	k.write(700, 5)

	if events := k.wait(); len(events) == 1 {
		fmt.Printf("1000 pipes registered, pipe 700 written: 1 event, for pipe %d\n", events[0])
	}

	byWait := k.examined
	k.scan()
	fmt.Printf("registrations examined: %d by the wait, %d by a scan\n", byWait, k.examined-byWait)
}


// turn is one iteration of an event loop: wait for ready pipes, then run the
// handler for each. It returns the pipes it handled.
func (k *kernel) turn(handle func(k *kernel, index int)) []int {
	events := k.wait()

	for _, index := range events {
		handle(k, index)
	}

	return events
}

// loop registers 4 pipes, writes to 2 of them, and runs an event loop whose
// handler reads at most 4 bytes for each event.
func loop() {
	k := newKernel(4)

	for index := range k.pipes {
		k.register(index, false)
	}

	k.write(2, 5)
	k.write(0, 3)

	handle := func(k *kernel, index int) { k.read(index, 4) }

	fmt.Printf("event loop: turn 1 handles %v, turn 2 handles %v, turn 3 handles %v\n",
		k.turn(handle), k.turn(handle), k.turn(handle))
}


func main() {
	level()
	edge()
	many()
	loop()
}

Listing 9.25 The complete eventloop, in Rust. code/rust/eventloop/src/main.rs

//! Models epoll: an interest list of registered pipes, a ready list that a
//! write fills, and a wait that examines only the ready list. It repeats the
//! measurements of epollfacts on the model and then runs a small event loop
//! on it.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p eventloop
//! ```

/// The read end of a pipe as epoll sees it: the bytes waiting to be read, and
/// its registration. `queued` is set while the pipe is on the ready list.
#[derive(Clone, Default)]
struct Pipe {
    buffered: usize,
    registered: bool,
    edge: bool,
    queued: bool,
}

/// Holds the pipes and one epoll instance. The registered pipes are the
/// interest list. `ready` is the ready list, in the order in which the pipes
/// became ready. `examined` counts the registrations that waits and scans
/// have looked at.
struct Kernel {
    pipes: Vec<Pipe>,
    ready: Vec<usize>,
    examined: usize,
}

impl Kernel {
    fn new(pipes: usize) -> Kernel {
        Kernel {
            pipes: vec![Pipe::default(); pipes],
            ready: Vec::new(),
            examined: 0,
        }
    }
}

impl Kernel {
    /// Adds a pipe to the interest list. A pipe that is already readable goes
    /// on the ready list at once.
    fn register(&mut self, index: usize, edge: bool) {
        let pipe = &mut self.pipes[index];

        pipe.registered = true;
        pipe.edge = edge;

        if pipe.buffered > 0 {
            self.enqueue(index);
        }
    }

    /// Puts a pipe on the ready list unless it is already there.
    fn enqueue(&mut self, index: usize) {
        if !self.pipes[index].queued {
            self.pipes[index].queued = true;
            self.ready.push(index);
        }
    }

    /// Adds bytes to a pipe. This is the moment the pipe becomes ready, and
    /// the writer puts it on the ready list.
    fn write(&mut self, index: usize, n: usize) {
        self.pipes[index].buffered += n;

        if self.pipes[index].registered {
            self.enqueue(index);
        }
    }

    /// Removes up to `n` bytes from a pipe and returns the number removed.
    fn read(&mut self, index: usize, n: usize) -> usize {
        let n = n.min(self.pipes[index].buffered);
        self.pipes[index].buffered -= n;

        n
    }
}

impl Kernel {
    /// Empties the ready list and returns the pipes on it that are still
    /// readable. A level-triggered pipe that is reported goes back on the
    /// list, so the next wait checks it again. An edge-triggered pipe does
    /// not, and is reported again only after another write.
    fn wait(&mut self) -> Vec<usize> {
        let queued = std::mem::take(&mut self.ready);
        let mut events = Vec::new();

        for index in queued {
            self.pipes[index].queued = false;
            self.examined += 1;

            if self.pipes[index].buffered == 0 {
                continue;
            }

            events.push(index);

            if !self.pipes[index].edge {
                self.enqueue(index);
            }
        }

        events
    }

    /// The method of select and poll: it examines every registered pipe on
    /// every call.
    fn scan(&mut self) -> Vec<usize> {
        let mut events = Vec::new();

        for (index, pipe) in self.pipes.iter().enumerate() {
            if !pipe.registered {
                continue;
            }

            self.examined += 1;

            if pipe.buffered > 0 {
                events.push(index);
            }
        }

        events
    }
}

/// Repeats the level-triggered measurement of epollfacts.
fn level() {
    let mut kernel = Kernel::new(3);

    for index in 0..3 {
        kernel.register(index, false);
    }

    println!(
        "3 pipes registered, none written: {} event(s)",
        kernel.wait().len()
    );

    kernel.write(1, 5);

    if let [index] = kernel.wait().as_slice() {
        println!("write to pipe 1: 1 event, for pipe {index}");
    }

    println!(
        "level-triggered, data not read, wait again: {} event(s)",
        kernel.wait().len()
    );

    kernel.read(1, 16);
    println!("after reading the data: {} event(s)", kernel.wait().len());
}

/// Repeats the edge-triggered measurement.
fn edge() {
    let mut kernel = Kernel::new(1);
    kernel.register(0, true);

    kernel.write(0, 5);
    let (first, second) = (kernel.wait().len(), kernel.wait().len());
    println!("edge-triggered, data not read: wait {first} event(s), wait again {second} event(s)");

    kernel.write(0, 5);
    println!(
        "edge-triggered, after another write: {} event(s)",
        kernel.wait().len()
    );
}

/// Compares the work of a wait with the work of a scan when 1000 pipes are
/// registered and 1 is ready.
fn many() {
    let mut kernel = Kernel::new(1000);

    for index in 0..1000 {
        kernel.register(index, false);
    }

    kernel.write(700, 5);

    if let [index] = kernel.wait().as_slice() {
        println!("1000 pipes registered, pipe 700 written: 1 event, for pipe {index}");
    }

    let by_wait = kernel.examined;
    kernel.scan();
    println!(
        "registrations examined: {by_wait} by the wait, {} by a scan",
        kernel.examined - by_wait
    );
}

impl Kernel {
    /// One iteration of an event loop: wait for ready pipes, then run the
    /// handler for each. Returns the pipes it handled.
    fn turn(&mut self, handle: fn(&mut Kernel, usize)) -> Vec<usize> {
        let events = self.wait();

        for &index in &events {
            handle(self, index);
        }

        events
    }
}

/// Formats pipe numbers the way Go prints a slice.
fn list(pipes: &[usize]) -> String {
    let words: Vec<String> = pipes.iter().map(usize::to_string).collect();

    format!("[{}]", words.join(" "))
}

/// Registers 4 pipes, writes to 2 of them, and runs an event loop whose
/// handler reads at most 4 bytes for each event.
fn event_loop() {
    let mut kernel = Kernel::new(4);

    for index in 0..4 {
        kernel.register(index, false);
    }

    kernel.write(2, 5);
    kernel.write(0, 3);

    let handle = |kernel: &mut Kernel, index: usize| {
        kernel.read(index, 4);
    };

    let turns = [
        kernel.turn(handle),
        kernel.turn(handle),
        kernel.turn(handle),
    ];

    println!(
        "event loop: turn 1 handles {}, turn 2 handles {}, turn 3 handles {}",
        list(&turns[0]),
        list(&turns[1]),
        list(&turns[2])
    );
}

fn main() {
    level();
    edge();
    many();
    event_loop();
}

References

• epoll(7), epoll_ctl(2) and epoll_wait(2), Linux manual pages: https://man7.org/linux/man-pages/man7/epoll.7.html
• M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010, chapter 63.
• fs/eventpoll.c, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/fs/eventpoll.c

TCP Sockets

A socket is the endpoint of a network connection, held by a process as a file descriptor. Behind the descriptor the kernel keeps the state of the connection: who the peer is, a buffer of data waiting to be sent, and a buffer of data received and not yet read. read and write work on a connected socket as they do on a pipe, and the same descriptor can be registered with epoll.

In this chapter, we will learn what the kernel does for a TCP connection between the calls a program makes. We will first measure a connection on the loopback interface with sockfacts. Then we will write sockmodel, a model of the receiving side: the table that finds the socket for an arriving packet, the accept queue of a listener, and the receive buffer that limits the sender.

From write to the network

A network carries packets: blocks of at most about 1500 bytes on Ethernet, each delivered independently, and each of which may be lost, duplicated or delivered out of order. A program wants something else: a stream of bytes that arrives complete and in order. The kernel’s network stack builds the second from the first, in layers.

An IP address names a machine, and a port is a 16-bit number that names a program on it. The loopback device, with the address 127.0.0.1, delivers packets to the same machine without any hardware, and the programs of this chapter use it.

The segments of a connection

TCP makes an unreliable network reliable by numbering. Every byte of a stream has a sequence number. Each packet of TCP, called a segment, says which bytes it carries. The receiver answers with an acknowledgement: the number of the next byte it expects, which confirms everything before it. The sender keeps each byte until it is acknowledged, and sends it again if no acknowledgement arrives in time. Out-of-order segments are put in order by their numbers, and duplicates are recognised and dropped.

An example with small numbers shows the whole mechanism. The client sends hello, 5 bytes, and its stream is currently at byte 1001.

The acknowledgement is a single number, and it confirms a whole range. This keeps the protocol simple: the receiver always says “the next byte I need is this one”, and whatever went wrong, the sender knows where to continue.

Before any data, the 2 sides have to agree on their starting sequence numbers. This is the handshake, 3 segments marked with the flags SYN and ACK.

1. SYN: the client proposes its starting number, 1000.
2. SYN and ACK: the server acknowledges it with 1001 and proposes its own, 5000.
3. ACK: the client acknowledges 5001. Both sides now know both numbers, and the connection is established.
4. The client sends 5 bytes, numbered 1001 to 1005.
5. The server acknowledges 1006 and reports its window, the space left in its receive buffer.
6. FIN: the client has nothing more to send. This closes its direction of the stream.
7. The server acknowledges. Its program’s next read returns 0. The server can still send, until it sends a FIN of its own.

All of this is done by the kernels. The programs call connect, write, read and close, and do not see a segment.

Setting up a connection

TCP gives 2 programs a reliable stream of bytes in each direction. A server and a client reach that state through different calls.

The handshake is carried out by the server’s kernel. The server program takes no part in it. An established connection waits in the accept queue of the listener until the program calls accept.

A connection is identified by 4 values: the local address, the local port, the remote address and the remote port. A listener and all the connections accepted from it have the same local port. An arriving packet, called a segment in TCP, is matched against the 4 values of the existing connections. A segment that matches none and asks to open a connection, a SYN, is given to the listener on its port.

The kernel’s structures

Behind a socket descriptor is a struct file (chapter 2) whose function table is the socket’s, and behind that a struct sock. For TCP the struct sock is the first part of a larger struct tcp_sock. These are fields from include/net/sock.h and include/linux/tcp.h of Linux 5.15.

struct sock_common {
        __be32 skc_daddr;                      /* remote IP address */
        __be32 skc_rcv_saddr;                  /* local IP address */
        __be16 skc_dport;                      /* remote port */
        __u16 skc_num;                         /* local port */
        volatile unsigned char skc_state;      /* TCP_LISTEN, TCP_ESTABLISHED, ... */
        /* ... */
};

struct sock {
        struct sock_common __sk_common;
        struct sk_buff_head sk_receive_queue;  /* segments received, not yet read */
        int sk_rcvbuf;                         /* limit of the receive queue, bytes */
        struct sk_buff_head sk_write_queue;    /* data written, not yet acknowledged */
        int sk_sndbuf;                         /* limit of the write queue, bytes */
        u32 sk_ack_backlog;                    /* connections waiting for accept */
        u32 sk_max_ack_backlog;                /* the backlog given to listen */
        /* ... */
};

struct tcp_sock {
        u32 rcv_nxt;                           /* next byte we expect to receive */
        u32 snd_nxt;                           /* next byte we will send */
        u32 snd_una;                           /* oldest byte sent and not acknowledged */
        u32 snd_wnd;                           /* the window the peer offered */
        u32 snd_cwnd;                          /* congestion window */
        /* ... */
};

The ss command prints these fields for the sockets of a system. Here a server listens on port 9099 and a client has connected to it over loopback:

$ ss -tln 'sport = :9099'
State  Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0      1          127.0.0.1:9099      0.0.0.0:*
$ ss -tn 'sport = :9099 or dport = :9099'
State Recv-Q Send-Q Local Address:Port  Peer Address:Port
ESTAB 0      0          127.0.0.1:9099     127.0.0.1:51252
ESTAB 0      0          127.0.0.1:51252    127.0.0.1:9099

The first command shows the listener. For a listener, Send-Q is the backlog, 1 here, and Recv-Q is the number of connections waiting for accept. The second shows the connection 2 times, because both ends are on this machine: the server’s socket with local port 9099, and the client’s with the port 51252 that the kernel chose for it. The 4 values of one line are the 4 values of the other, exchanged.

Measuring a connection

sockfacts makes its connections over the loopback address 127.0.0.1, with both ends in the same process. The Go version calls the socket system calls through golang.org/x/sys/unix. The Rust version uses std::net and the nix crate with the socket feature for socket options.

$ cargo new sockfacts
$ cargo add -p sockfacts nix --features socket

listen binds to port 0, which asks the kernel for any free port.

Listing 10.1 The imports of sockfacts, in Go. code/go/sockfacts/main.go

import (
	"errors"
	"fmt"
	"log"

	"golang.org/x/sys/unix"
)

Listing 10.2 The imports of sockfacts, in Rust. code/rust/sockfacts/src/main.rs

use std::io::{self, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::os::fd::AsRawFd;

use nix::errno::Errno;
use nix::sys::socket::{setsockopt, sockopt};

Listing 10.3 A listener and a client on the loopback interface, in Go. code/go/sockfacts/main.go

// listen returns a TCP socket that listens on a free port of 127.0.0.1, and
// its address.
func listen() (int, *unix.SockaddrInet4, error) {
	listener, err := unix.Socket(unix.AF_INET, unix.SOCK_STREAM|unix.SOCK_CLOEXEC, 0)

	if err != nil {
		return 0, nil, err
	}

	if err := unix.Bind(listener, &unix.SockaddrInet4{Addr: [4]byte{127, 0, 0, 1}}); err != nil {
		return 0, nil, err
	}

	if err := unix.Listen(listener, 16); err != nil {
		return 0, nil, err
	}

	address, err := unix.Getsockname(listener)

	if err != nil {
		return 0, nil, err
	}

	return listener, address.(*unix.SockaddrInet4), nil
}

// connect returns a TCP socket connected to an address.
func connect(address *unix.SockaddrInet4) (int, error) {
	client, err := unix.Socket(unix.AF_INET, unix.SOCK_STREAM|unix.SOCK_CLOEXEC, 0)

	if err != nil {
		return 0, err
	}

	return client, unix.Connect(client, address)
}

// name returns the symbolic name of a system call error, such as EPIPE.
func name(err error) string {
	var errno unix.Errno

	if errors.As(err, &errno) {
		return unix.ErrnoName(errno)
	}

	return fmt.Sprint(err)
}

// port returns the port of an address.
func port(address unix.Sockaddr, err error) int {
	if inet, ok := address.(*unix.SockaddrInet4); ok && err == nil {
		return inet.Port
	}

	return 0
}

Listing 10.4 A listener and a client on the loopback interface, in Rust. code/rust/sockfacts/src/main.rs

/// Returns a TCP socket that listens on a free port of 127.0.0.1.
fn listen() -> io::Result<TcpListener> {
    TcpListener::bind("127.0.0.1:0")
}

/// Returns a TCP socket connected to the address of a listener.
fn connect(listener: &TcpListener) -> io::Result<TcpStream> {
    TcpStream::connect(listener.local_addr()?)
}

/// Returns the symbolic name of a system call error, such as `EPIPE`.
fn name(error: &io::Error) -> String {
    match error.raw_os_error() {
        Some(code) => format!("{:?}", Errno::from_raw(code)),
        None => error.to_string(),
    }
}

establish calls connect before the server has called accept. Then it accepts and compares the addresses of the ends.

Listing 10.5 Connecting before accept, and the ports of both ends, in Go. code/go/sockfacts/main.go

// establish connects before the server calls accept, and compares the
// addresses of both ends.
func establish() error {
	listener, address, err := listen()

	if err != nil {
		return err
	}

	defer unix.Close(listener)

	client, err := connect(address)
	fmt.Printf("connect before the server calls accept: succeeded %v\n", err == nil)

	if err != nil {
		return err
	}

	defer unix.Close(client)

	server, _, err := unix.Accept(listener)

	if err != nil {
		return err
	}

	defer unix.Close(server)

	fmt.Printf("accept returns a new descriptor: %v, on the listener's port: %v\n",
		server != listener, port(unix.Getsockname(server)) == address.Port)
	fmt.Printf("the client's local port is the server's peer port: %v\n",
		port(unix.Getsockname(client)) == port(unix.Getpeername(server)))

	return stream(client, server)
}

Listing 10.6 Connecting before accept, and the ports of both ends, in Rust. code/rust/sockfacts/src/main.rs

/// Connects before the server calls accept, and compares the addresses of
/// both ends.
fn establish() -> io::Result<()> {
    let listener = listen()?;

    let client = connect(&listener);
    println!(
        "connect before the server calls accept: succeeded {}",
        client.is_ok()
    );

    let client = client?;
    let (server, _) = listener.accept()?;

    println!(
        "accept returns a new descriptor: {}, on the listener's port: {}",
        server.as_raw_fd() != listener.as_raw_fd(),
        server.local_addr()?.port() == listener.local_addr()?.port()
    );
    println!(
        "the client's local port is the server's peer port: {}",
        client.local_addr()?.port() == server.peer_addr()?.port()
    );

    stream(client, server)
}

stream sends ab and cd with separate calls to write, and the server reads 3 bytes and then 1. After that the client closes its socket. The server reads again, and then writes until a write fails.

Listing 10.7 A stream of bytes, and the end of the stream, in Go. code/go/sockfacts/main.go

// readFull reads until buf is full.
func readFull(fd int, buf []byte) error {
	for filled := 0; filled < len(buf); {
		n, err := unix.Read(fd, buf[filled:])

		if err != nil {
			return err
		}

		if n == 0 {
			return errors.New("the stream ended early")
		}

		filled += n
	}

	return nil
}

// stream writes "ab" and "cd" with 2 calls and reads 3 bytes and then 1.
// Then the client closes, and the server reads and writes again.
func stream(client, server int) error {
	for _, part := range []string{"ab", "cd"} {
		if _, err := unix.Write(client, []byte(part)); err != nil {
			return err
		}
	}

	first, second := make([]byte, 3), make([]byte, 1)

	if err := readFull(server, first); err != nil {
		return err
	}

	if err := readFull(server, second); err != nil {
		return err
	}

	fmt.Printf("writes of \"ab\" and \"cd\", reads of 3 bytes and 1 byte: %q and %q\n", first, second)

	unix.Close(client)

	n, err := unix.Read(server, first)

	if err != nil {
		return err
	}

	fmt.Printf("the client closed: read returns %d\n", n)

	for err == nil {
		_, err = unix.Write(server, []byte("x"))
	}

	fmt.Printf("writes to the closed peer: the first is accepted, a later one fails with %s\n", name(err))

	return nil
}

Listing 10.8 A stream of bytes, and the end of the stream, in Rust. code/rust/sockfacts/src/main.rs

/// Writes "ab" and "cd" with 2 calls and reads 3 bytes and then 1. Then the
/// client closes, and the server reads and writes again.
fn stream(mut client: TcpStream, mut server: TcpStream) -> io::Result<()> {
    for part in ["ab", "cd"] {
        client.write_all(part.as_bytes())?;
    }

    let (mut first, mut second) = ([0u8; 3], [0u8; 1]);

    server.read_exact(&mut first)?;
    server.read_exact(&mut second)?;

    println!(
        "writes of \"ab\" and \"cd\", reads of 3 bytes and 1 byte: {:?} and {:?}",
        String::from_utf8_lossy(&first),
        String::from_utf8_lossy(&second)
    );

    drop(client);

    let n = server.read(&mut first)?;
    println!("the client closed: read returns {n}");

    let error = loop {
        if let Err(error) = server.write(b"x") {
            break error;
        }
    };

    println!(
        "writes to the closed peer: the first is accepted, a later one fails with {}",
        name(&error)
    );

    Ok(())
}

flow reduces the buffers of a connection, so that the numbers are small, and writes to the client socket in non-blocking mode until the kernel refuses a write. The server does not read during that time. Then the server reads everything, and the client tries again.

Listing 10.9 A sender whose peer does not read, in Go. code/go/sockfacts/main.go

// fill writes blocks to a non-blocking socket until the kernel refuses one
// and returns the number of bytes accepted.
func fill(fd int) (int, error) {
	block := make([]byte, 1024)
	total := 0

	for {
		n, err := unix.Write(fd, block)

		if errors.Is(err, unix.EAGAIN) {
			return total, nil
		}

		if err != nil {
			return total, err
		}

		total += n
	}
}

// flow gives a connection small buffers, fills it while the server does not
// read, lets the server read everything, and fills it again.
func flow() error {
	listener, address, err := listen()

	if err != nil {
		return err
	}

	defer unix.Close(listener)

	if err := unix.SetsockoptInt(listener, unix.SOL_SOCKET, unix.SO_RCVBUF, 4096); err != nil {
		return err
	}

	client, err := connect(address)

	if err != nil {
		return err
	}

	defer unix.Close(client)

	if err := unix.SetsockoptInt(client, unix.SOL_SOCKET, unix.SO_SNDBUF, 4096); err != nil {
		return err
	}

	server, _, err := unix.Accept(listener)

	if err != nil {
		return err
	}

	defer unix.Close(server)

	if err := unix.SetNonblock(client, true); err != nil {
		return err
	}

	sent, err := fill(client)

	if err != nil {
		return err
	}

	fmt.Printf("the server does not read: the client is stopped after %d bytes\n", sent)

	if err := readFull(server, make([]byte, sent)); err != nil {
		return err
	}

	again, err := fill(client)

	if err != nil {
		return err
	}

	fmt.Printf("the server has read everything: the client can write again: %v\n", again > 0)

	return nil
}

Listing 10.10 A sender whose peer does not read, in Rust. code/rust/sockfacts/src/main.rs

/// Writes blocks to a non-blocking socket until the kernel refuses one and
/// returns the number of bytes accepted.
fn fill(mut stream: &TcpStream) -> io::Result<usize> {
    let block = [0u8; 1024];
    let mut total = 0;

    loop {
        match stream.write(&block) {
            Ok(n) => total += n,
            Err(error) if error.kind() == io::ErrorKind::WouldBlock => return Ok(total),
            Err(error) => return Err(error),
        }
    }
}

/// Gives a connection small buffers, fills it while the server does not read,
/// lets the server read everything, and fills it again.
fn flow() -> io::Result<()> {
    let listener = listen()?;
    setsockopt(&listener, sockopt::RcvBuf, &4096)?;

    let client = connect(&listener)?;
    setsockopt(&client, sockopt::SndBuf, &4096)?;

    let (mut server, _) = listener.accept()?;

    client.set_nonblocking(true)?;

    let sent = fill(&client)?;
    println!("the server does not read: the client is stopped after {sent} bytes");

    server.read_exact(&mut vec![0u8; sent])?;

    let again = fill(&client)?;
    println!(
        "the server has read everything: the client can write again: {}",
        again > 0
    );

    Ok(())
}

connect before the server calls accept: succeeded true
accept returns a new descriptor: true, on the listener's port: true
the client's local port is the server's peer port: true
writes of "ab" and "cd", reads of 3 bytes and 1 byte: "abc" and "d"
the client closed: read returns 0
writes to the closed peer: the first is accepted, a later one fails with EPIPE
the server does not read: the client is stopped after 7168 bytes
the server has read everything: the client can write again: true

Row 4 has a practical consequence. A protocol built on TCP has to mark the boundaries of its messages itself, with a length before each message or with a delimiter such as a newline, and a reader has to be prepared to receive part of a message or several messages in a single read.

Buffers and the window

A write on a socket copies the data into the socket’s send buffer and returns. The kernel transmits from that buffer and keeps each byte until the peer has acknowledged it. On the other side, arriving data is placed in the receive buffer until the program reads it.

The 2 buffers are connected by the window. With every acknowledgement, the receiver reports how much free space its receive buffer has, and the sender transmits no more than that. When the receiving program stops reading, the receive buffer fills, the window becomes 0, the sender’s kernel stops transmitting, the send buffer fills, and write blocks. This is the flow control of a pipe, extended across a network. The 7168 bytes of the measurement are the contents of both buffers at that point.

Modelling the receiving side

The model covers the receiving half of TCP and omits transmission, acknowledgements and retransmission.

Listing 10.11 The imports of sockmodel, in Go. code/go/sockmodel/main.go

import "fmt"

Listing 10.12 The imports of sockmodel, in Rust. code/rust/sockmodel/src/main.rs

use std::collections::{HashMap, VecDeque};
use std::fmt;

Listing 10.13 Endpoints, sockets, listeners and segments, in Go. code/go/sockmodel/main.go

// An endpoint is an IP address and a port.
type endpoint struct {
	address string
	port    int
}

func (e endpoint) String() string {
	return fmt.Sprintf("%s:%d", e.address, e.port)
}

// A socket is one established connection: the bytes received and not yet
// read, the size of the receive buffer, and whether the sender has finished.
type socket struct {
	received []byte
	capacity int
	finished bool
}

// window is the free space of the receive buffer: the number of bytes the
// sender may still send.
func (s *socket) window() int {
	return s.capacity - len(s.received)
}

// A key identifies a connection on this machine: the remote endpoint and the
// local port. With the local address, which the model leaves out, it forms
// the 4 values that identify a TCP connection.
type key struct {
	remote endpoint
	port   int
}

// A listener accepts connections on a port. queue holds the connections that
// are established and have not been returned by accept.
type listener struct {
	backlog int
	queue   []key
}

// A stack is the TCP state of a machine.
type stack struct {
	listeners   map[int]*listener
	connections map[key]*socket
	bufferSize  int
}

// A segment is a packet of TCP. syn opens a connection and fin ends the
// sender's stream.
type segment struct {
	from     endpoint
	port     int
	syn, fin bool
	data     string
}

Listing 10.14 Endpoints, sockets, listeners and segments, in Rust. code/rust/sockmodel/src/main.rs

/// An IP address and a port.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
struct Endpoint {
    address: &'static str,
    port: u16,
}

impl fmt::Display for Endpoint {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "{}:{}", self.address, self.port)
    }
}

/// One established connection: the bytes received and not yet read, the size
/// of the receive buffer, and whether the sender has finished.
struct Socket {
    received: VecDeque<u8>,
    capacity: usize,
    finished: bool,
}

impl Socket {
    /// The free space of the receive buffer: the number of bytes the sender
    /// may still send.
    fn window(&self) -> usize {
        self.capacity - self.received.len()
    }
}

/// Identifies a connection on this machine: the remote endpoint and the local
/// port. With the local address, which the model leaves out, it forms the 4
/// values that identify a TCP connection.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
struct Key {
    remote: Endpoint,
    port: u16,
}

/// Accepts connections on a port. `queue` holds the connections that are
/// established and have not been returned by accept.
struct Listener {
    backlog: usize,
    queue: VecDeque<Key>,
}

/// The TCP state of a machine.
struct Stack {
    listeners: HashMap<u16, Listener>,
    connections: HashMap<Key, Socket>,
    buffer_size: usize,
}

/// A packet of TCP. `syn` opens a connection and `fin` ends the sender's
/// stream.
#[derive(Default)]
struct Segment {
    syn: bool,
    fin: bool,
    data: &'static str,
}

receive is the lookup of figure 10.4. The table of connections is consulted first, which is why data for an established connection reaches its own socket while a listener exists on the same port. A SYN for a full accept queue is dropped without an answer. The client’s kernel sends it again after a timeout, which gives the server time to call accept.

Listing 10.15 Finding the socket for a segment, in Go. code/go/sockmodel/main.go

// receive handles an arriving segment and describes the result. A segment
// belongs to the connection with its remote endpoint and local port. Only
// when no such connection exists, and the segment is a SYN, is the listener
// on the port consulted.
func (st *stack) receive(seg segment) string {
	id := key{remote: seg.from, port: seg.port}

	if s, ok := st.connections[id]; ok {
		return s.deliver(seg)
	}

	if !seg.syn {
		return "reset, no connection"
	}

	l, ok := st.listeners[seg.port]

	if !ok {
		return "reset, no listener on the port"
	}

	if len(l.queue) == l.backlog {
		return "dropped, the accept queue is full"
	}

	st.connections[id] = &socket{capacity: st.bufferSize}
	l.queue = append(l.queue, id)

	return fmt.Sprintf("established, accept queue %d", len(l.queue))
}

// deliver puts the data of a segment into the receive buffer, up to the
// window, and records a FIN.
func (s *socket) deliver(seg segment) string {
	if seg.fin {
		s.finished = true

		return "the sender's stream has ended"
	}

	accepted := min(len(seg.data), s.window())
	s.received = append(s.received, seg.data[:accepted]...)

	return fmt.Sprintf("accepted %d, window %d", accepted, s.window())
}

Listing 10.16 Finding the socket for a segment, in Rust. code/rust/sockmodel/src/main.rs

impl Stack {
    /// Handles a segment that arrives from `from` for local port `port`, and
    /// describes the result. A segment belongs to the connection with its
    /// remote endpoint and local port. Only when no such connection exists,
    /// and the segment is a SYN, is the listener on the port consulted.
    fn receive(&mut self, from: Endpoint, port: u16, segment: &Segment) -> String {
        let id = Key { remote: from, port };

        if let Some(socket) = self.connections.get_mut(&id) {
            return socket.deliver(segment);
        }

        if !segment.syn {
            return "reset, no connection".into();
        }

        let Some(listener) = self.listeners.get_mut(&port) else {
            return "reset, no listener on the port".into();
        };

        if listener.queue.len() == listener.backlog {
            return "dropped, the accept queue is full".into();
        }

        let socket = Socket {
            received: VecDeque::new(),
            capacity: self.buffer_size,
            finished: false,
        };

        self.connections.insert(id, socket);
        listener.queue.push_back(id);

        format!("established, accept queue {}", listener.queue.len())
    }
}

impl Socket {
    /// Puts the data of a segment into the receive buffer, up to the window,
    /// and records a FIN.
    fn deliver(&mut self, segment: &Segment) -> String {
        if segment.fin {
            self.finished = true;

            return "the sender's stream has ended".into();
        }

        let accepted = segment.data.len().min(self.window());
        self.received.extend(&segment.data.as_bytes()[..accepted]);

        format!("accepted {accepted}, window {}", self.window())
    }
}

deliver accepts data up to the window. window is the free space of the receive buffer, the value a real receiver reports to the sender.

accept takes connections in the order they were established. read returns the end of the stream only when the buffer is empty and the FIN, the segment that ends the sender’s stream, has arrived, so data that was sent before a close is not lost.

Listing 10.17 Accepting a connection, and reading from it, in Go. code/go/sockmodel/main.go

// accept removes the oldest connection from the accept queue of a port.
func (st *stack) accept(port int) (key, bool) {
	l, ok := st.listeners[port]

	if !ok || len(l.queue) == 0 {
		return key{}, false
	}

	id := l.queue[0]
	l.queue = l.queue[1:]

	return id, true
}

// read removes up to n bytes from the receive buffer. It returns 0 bytes
// with ended set when the buffer is empty and a FIN has arrived.
func (s *socket) read(n int) (data string, ended bool) {
	n = min(n, len(s.received))
	data, s.received = string(s.received[:n]), s.received[n:]

	return data, n == 0 && s.finished
}

Listing 10.18 Accepting a connection, and reading from it, in Rust. code/rust/sockmodel/src/main.rs

impl Stack {
    /// Removes the oldest connection from the accept queue of a port.
    fn accept(&mut self, port: u16) -> Option<Key> {
        self.listeners.get_mut(&port)?.queue.pop_front()
    }
}

impl Socket {
    /// Removes up to `n` bytes from the receive buffer. It returns 0 bytes
    /// with the second value set when the buffer is empty and a FIN has
    /// arrived.
    fn read(&mut self, n: usize) -> (String, bool) {
        let n = n.min(self.received.len());
        let data: Vec<u8> = self.received.drain(..n).collect();

        (
            String::from_utf8_lossy(&data).into_owned(),
            n == 0 && self.finished,
        )
    }
}

Listing 10.19 The trace, in Go. code/go/sockmodel/main.go

func main() {
	st := &stack{
		listeners:   map[int]*listener{80: {backlog: 2}},
		connections: map[key]*socket{},
		bufferSize:  8,
	}

	a, b, c := endpoint{"10.0.0.1", 5000}, endpoint{"10.0.0.2", 5000}, endpoint{"10.0.0.3", 5000}

	for _, seg := range []segment{{from: a, port: 80, syn: true}, {from: b, port: 80, syn: true},
		{from: c, port: 80, syn: true}, {from: c, port: 81, syn: true}} {
		fmt.Printf("SYN from %s to port %d: %s\n", seg.from, seg.port, st.receive(seg))
	}

	if id, ok := st.accept(80); ok {
		fmt.Printf("accept on port 80: the connection from %s\n", id.remote)
	}

	for _, from := range []endpoint{b, a, b, c} {
		fmt.Printf("5 bytes from %s to port 80: %s\n", from, st.receive(segment{from: from, port: 80, data: "hello"}))
	}

	conn, ok := st.connections[key{remote: b, port: 80}]

	if !ok {
		return
	}

	data, ended := conn.read(4)
	fmt.Printf("read of 4 bytes on the connection from %s: %q, window %d\n", b, data, conn.window())

	fmt.Printf("FIN from %s: %s\n", b, st.receive(segment{from: b, port: 80, fin: true}))

	data, ended = conn.read(16)
	fmt.Printf("read: %q, stream ended %v\n", data, ended)

	data, ended = conn.read(16)
	fmt.Printf("read: %q, stream ended %v\n", data, ended)
}

Listing 10.20 The trace, in Rust. code/rust/sockmodel/src/main.rs

fn endpoint(address: &'static str) -> Endpoint {
    Endpoint {
        address,
        port: 5000,
    }
}

fn main() {
    let listener = Listener {
        backlog: 2,
        queue: VecDeque::new(),
    };

    let mut stack = Stack {
        listeners: HashMap::from([(80, listener)]),
        connections: HashMap::new(),
        buffer_size: 8,
    };

    let (a, b, c) = (
        endpoint("10.0.0.1"),
        endpoint("10.0.0.2"),
        endpoint("10.0.0.3"),
    );

    let syn = Segment {
        syn: true,
        ..Segment::default()
    };

    for (from, port) in [(a, 80), (b, 80), (c, 80), (c, 81)] {
        let result = stack.receive(from, port, &syn);
        println!("SYN from {from} to port {port}: {result}");
    }

    if let Some(id) = stack.accept(80) {
        println!("accept on port 80: the connection from {}", id.remote);
    }

    let hello = Segment {
        data: "hello",
        ..Segment::default()
    };

    for from in [b, a, b, c] {
        let result = stack.receive(from, 80, &hello);
        println!("5 bytes from {from} to port 80: {result}");
    }

    let id = Key {
        remote: b,
        port: 80,
    };

    if let Some(socket) = stack.connections.get_mut(&id) {
        let (data, _) = socket.read(4);
        println!(
            "read of 4 bytes on the connection from {b}: {data:?}, window {}",
            socket.window()
        );
    }

    let fin = Segment {
        fin: true,
        ..Segment::default()
    };

    println!("FIN from {b}: {}", stack.receive(b, 80, &fin));

    if let Some(socket) = stack.connections.get_mut(&id) {
        for _ in 0..2 {
            let (data, ended) = socket.read(16);
            println!("read: {data:?}, stream ended {ended}");
        }
    }
}

SYN from 10.0.0.1:5000 to port 80: established, accept queue 1
SYN from 10.0.0.2:5000 to port 80: established, accept queue 2
SYN from 10.0.0.3:5000 to port 80: dropped, the accept queue is full
SYN from 10.0.0.3:5000 to port 81: reset, no listener on the port
accept on port 80: the connection from 10.0.0.1:5000
5 bytes from 10.0.0.2:5000 to port 80: accepted 5, window 3
5 bytes from 10.0.0.1:5000 to port 80: accepted 5, window 3
5 bytes from 10.0.0.2:5000 to port 80: accepted 3, window 0
5 bytes from 10.0.0.3:5000 to port 80: reset, no connection
read of 4 bytes on the connection from 10.0.0.2:5000: "hell", window 4
FIN from 10.0.0.2:5000: the sender's stream has ended
read: "ohel", stream ended false
read: "", stream ended true

Inspecting sockets on a live system

Recv-Q on an established connection is data that has arrived and that the program has not read. A value that stays high means the program is slow to read, and the sender is being held back by the window.

The complete programs

Listing 10.21 The complete sockfacts, in Go. code/go/sockfacts/main.go

// Command sockfacts measures TCP sockets on the loopback interface: when a
// connection is established, what identifies it, how written bytes arrive,
// what stops a writer whose peer does not read, and how each side learns
// that the other has closed.
//
// Run it from code/go with:
//
//	go run ./sockfacts
package main

import (
	"errors"
	"fmt"
	"log"

	"golang.org/x/sys/unix"
)


// listen returns a TCP socket that listens on a free port of 127.0.0.1, and
// its address.
func listen() (int, *unix.SockaddrInet4, error) {
	listener, err := unix.Socket(unix.AF_INET, unix.SOCK_STREAM|unix.SOCK_CLOEXEC, 0)

	if err != nil {
		return 0, nil, err
	}

	if err := unix.Bind(listener, &unix.SockaddrInet4{Addr: [4]byte{127, 0, 0, 1}}); err != nil {
		return 0, nil, err
	}

	if err := unix.Listen(listener, 16); err != nil {
		return 0, nil, err
	}

	address, err := unix.Getsockname(listener)

	if err != nil {
		return 0, nil, err
	}

	return listener, address.(*unix.SockaddrInet4), nil
}

// connect returns a TCP socket connected to an address.
func connect(address *unix.SockaddrInet4) (int, error) {
	client, err := unix.Socket(unix.AF_INET, unix.SOCK_STREAM|unix.SOCK_CLOEXEC, 0)

	if err != nil {
		return 0, err
	}

	return client, unix.Connect(client, address)
}

// name returns the symbolic name of a system call error, such as EPIPE.
func name(err error) string {
	var errno unix.Errno

	if errors.As(err, &errno) {
		return unix.ErrnoName(errno)
	}

	return fmt.Sprint(err)
}

// port returns the port of an address.
func port(address unix.Sockaddr, err error) int {
	if inet, ok := address.(*unix.SockaddrInet4); ok && err == nil {
		return inet.Port
	}

	return 0
}


// establish connects before the server calls accept, and compares the
// addresses of both ends.
func establish() error {
	listener, address, err := listen()

	if err != nil {
		return err
	}

	defer unix.Close(listener)

	client, err := connect(address)
	fmt.Printf("connect before the server calls accept: succeeded %v\n", err == nil)

	if err != nil {
		return err
	}

	defer unix.Close(client)

	server, _, err := unix.Accept(listener)

	if err != nil {
		return err
	}

	defer unix.Close(server)

	fmt.Printf("accept returns a new descriptor: %v, on the listener's port: %v\n",
		server != listener, port(unix.Getsockname(server)) == address.Port)
	fmt.Printf("the client's local port is the server's peer port: %v\n",
		port(unix.Getsockname(client)) == port(unix.Getpeername(server)))

	return stream(client, server)
}


// readFull reads until buf is full.
func readFull(fd int, buf []byte) error {
	for filled := 0; filled < len(buf); {
		n, err := unix.Read(fd, buf[filled:])

		if err != nil {
			return err
		}

		if n == 0 {
			return errors.New("the stream ended early")
		}

		filled += n
	}

	return nil
}

// stream writes "ab" and "cd" with 2 calls and reads 3 bytes and then 1.
// Then the client closes, and the server reads and writes again.
func stream(client, server int) error {
	for _, part := range []string{"ab", "cd"} {
		if _, err := unix.Write(client, []byte(part)); err != nil {
			return err
		}
	}

	first, second := make([]byte, 3), make([]byte, 1)

	if err := readFull(server, first); err != nil {
		return err
	}

	if err := readFull(server, second); err != nil {
		return err
	}

	fmt.Printf("writes of \"ab\" and \"cd\", reads of 3 bytes and 1 byte: %q and %q\n", first, second)

	unix.Close(client)

	n, err := unix.Read(server, first)

	if err != nil {
		return err
	}

	fmt.Printf("the client closed: read returns %d\n", n)

	for err == nil {
		_, err = unix.Write(server, []byte("x"))
	}

	fmt.Printf("writes to the closed peer: the first is accepted, a later one fails with %s\n", name(err))

	return nil
}


// fill writes blocks to a non-blocking socket until the kernel refuses one
// and returns the number of bytes accepted.
func fill(fd int) (int, error) {
	block := make([]byte, 1024)
	total := 0

	for {
		n, err := unix.Write(fd, block)

		if errors.Is(err, unix.EAGAIN) {
			return total, nil
		}

		if err != nil {
			return total, err
		}

		total += n
	}
}

// flow gives a connection small buffers, fills it while the server does not
// read, lets the server read everything, and fills it again.
func flow() error {
	listener, address, err := listen()

	if err != nil {
		return err
	}

	defer unix.Close(listener)

	if err := unix.SetsockoptInt(listener, unix.SOL_SOCKET, unix.SO_RCVBUF, 4096); err != nil {
		return err
	}

	client, err := connect(address)

	if err != nil {
		return err
	}

	defer unix.Close(client)

	if err := unix.SetsockoptInt(client, unix.SOL_SOCKET, unix.SO_SNDBUF, 4096); err != nil {
		return err
	}

	server, _, err := unix.Accept(listener)

	if err != nil {
		return err
	}

	defer unix.Close(server)

	if err := unix.SetNonblock(client, true); err != nil {
		return err
	}

	sent, err := fill(client)

	if err != nil {
		return err
	}

	fmt.Printf("the server does not read: the client is stopped after %d bytes\n", sent)

	if err := readFull(server, make([]byte, sent)); err != nil {
		return err
	}

	again, err := fill(client)

	if err != nil {
		return err
	}

	fmt.Printf("the server has read everything: the client can write again: %v\n", again > 0)

	return nil
}


func main() {
	for _, experiment := range []func() error{establish, flow} {
		if err := experiment(); err != nil {
			log.Fatal(err)
		}
	}
}

Listing 10.22 The complete sockfacts, in Rust. code/rust/sockfacts/src/main.rs

//! Measures TCP sockets on the loopback interface: when a connection is
//! established, what identifies it, how written bytes arrive, what stops a
//! writer whose peer does not read, and how each side learns that the other
//! has closed.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p sockfacts
//! ```

use std::io::{self, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::os::fd::AsRawFd;

use nix::errno::Errno;
use nix::sys::socket::{setsockopt, sockopt};

/// Returns a TCP socket that listens on a free port of 127.0.0.1.
fn listen() -> io::Result<TcpListener> {
    TcpListener::bind("127.0.0.1:0")
}

/// Returns a TCP socket connected to the address of a listener.
fn connect(listener: &TcpListener) -> io::Result<TcpStream> {
    TcpStream::connect(listener.local_addr()?)
}

/// Returns the symbolic name of a system call error, such as `EPIPE`.
fn name(error: &io::Error) -> String {
    match error.raw_os_error() {
        Some(code) => format!("{:?}", Errno::from_raw(code)),
        None => error.to_string(),
    }
}

/// Connects before the server calls accept, and compares the addresses of
/// both ends.
fn establish() -> io::Result<()> {
    let listener = listen()?;

    let client = connect(&listener);
    println!(
        "connect before the server calls accept: succeeded {}",
        client.is_ok()
    );

    let client = client?;
    let (server, _) = listener.accept()?;

    println!(
        "accept returns a new descriptor: {}, on the listener's port: {}",
        server.as_raw_fd() != listener.as_raw_fd(),
        server.local_addr()?.port() == listener.local_addr()?.port()
    );
    println!(
        "the client's local port is the server's peer port: {}",
        client.local_addr()?.port() == server.peer_addr()?.port()
    );

    stream(client, server)
}

/// Writes "ab" and "cd" with 2 calls and reads 3 bytes and then 1. Then the
/// client closes, and the server reads and writes again.
fn stream(mut client: TcpStream, mut server: TcpStream) -> io::Result<()> {
    for part in ["ab", "cd"] {
        client.write_all(part.as_bytes())?;
    }

    let (mut first, mut second) = ([0u8; 3], [0u8; 1]);

    server.read_exact(&mut first)?;
    server.read_exact(&mut second)?;

    println!(
        "writes of \"ab\" and \"cd\", reads of 3 bytes and 1 byte: {:?} and {:?}",
        String::from_utf8_lossy(&first),
        String::from_utf8_lossy(&second)
    );

    drop(client);

    let n = server.read(&mut first)?;
    println!("the client closed: read returns {n}");

    let error = loop {
        if let Err(error) = server.write(b"x") {
            break error;
        }
    };

    println!(
        "writes to the closed peer: the first is accepted, a later one fails with {}",
        name(&error)
    );

    Ok(())
}

/// Writes blocks to a non-blocking socket until the kernel refuses one and
/// returns the number of bytes accepted.
fn fill(mut stream: &TcpStream) -> io::Result<usize> {
    let block = [0u8; 1024];
    let mut total = 0;

    loop {
        match stream.write(&block) {
            Ok(n) => total += n,
            Err(error) if error.kind() == io::ErrorKind::WouldBlock => return Ok(total),
            Err(error) => return Err(error),
        }
    }
}

/// Gives a connection small buffers, fills it while the server does not read,
/// lets the server read everything, and fills it again.
fn flow() -> io::Result<()> {
    let listener = listen()?;
    setsockopt(&listener, sockopt::RcvBuf, &4096)?;

    let client = connect(&listener)?;
    setsockopt(&client, sockopt::SndBuf, &4096)?;

    let (mut server, _) = listener.accept()?;

    client.set_nonblocking(true)?;

    let sent = fill(&client)?;
    println!("the server does not read: the client is stopped after {sent} bytes");

    server.read_exact(&mut vec![0u8; sent])?;

    let again = fill(&client)?;
    println!(
        "the server has read everything: the client can write again: {}",
        again > 0
    );

    Ok(())
}

fn main() -> io::Result<()> {
    establish()?;
    flow()
}

Listing 10.23 The complete sockmodel, in Go. code/go/sockmodel/main.go

// Command sockmodel models the receiving side of TCP in the kernel: a table
// of connections keyed by addresses and ports, listeners with an accept
// queue of limited length, and a receive buffer whose free space is the
// window offered to the sender.
//
// Run it from code/go with:
//
//	go run ./sockmodel
package main

import "fmt"


// An endpoint is an IP address and a port.
type endpoint struct {
	address string
	port    int
}

func (e endpoint) String() string {
	return fmt.Sprintf("%s:%d", e.address, e.port)
}

// A socket is one established connection: the bytes received and not yet
// read, the size of the receive buffer, and whether the sender has finished.
type socket struct {
	received []byte
	capacity int
	finished bool
}

// window is the free space of the receive buffer: the number of bytes the
// sender may still send.
func (s *socket) window() int {
	return s.capacity - len(s.received)
}

// A key identifies a connection on this machine: the remote endpoint and the
// local port. With the local address, which the model leaves out, it forms
// the 4 values that identify a TCP connection.
type key struct {
	remote endpoint
	port   int
}

// A listener accepts connections on a port. queue holds the connections that
// are established and have not been returned by accept.
type listener struct {
	backlog int
	queue   []key
}

// A stack is the TCP state of a machine.
type stack struct {
	listeners   map[int]*listener
	connections map[key]*socket
	bufferSize  int
}

// A segment is a packet of TCP. syn opens a connection and fin ends the
// sender's stream.
type segment struct {
	from     endpoint
	port     int
	syn, fin bool
	data     string
}


// receive handles an arriving segment and describes the result. A segment
// belongs to the connection with its remote endpoint and local port. Only
// when no such connection exists, and the segment is a SYN, is the listener
// on the port consulted.
func (st *stack) receive(seg segment) string {
	id := key{remote: seg.from, port: seg.port}

	if s, ok := st.connections[id]; ok {
		return s.deliver(seg)
	}

	if !seg.syn {
		return "reset, no connection"
	}

	l, ok := st.listeners[seg.port]

	if !ok {
		return "reset, no listener on the port"
	}

	if len(l.queue) == l.backlog {
		return "dropped, the accept queue is full"
	}

	st.connections[id] = &socket{capacity: st.bufferSize}
	l.queue = append(l.queue, id)

	return fmt.Sprintf("established, accept queue %d", len(l.queue))
}

// deliver puts the data of a segment into the receive buffer, up to the
// window, and records a FIN.
func (s *socket) deliver(seg segment) string {
	if seg.fin {
		s.finished = true

		return "the sender's stream has ended"
	}

	accepted := min(len(seg.data), s.window())
	s.received = append(s.received, seg.data[:accepted]...)

	return fmt.Sprintf("accepted %d, window %d", accepted, s.window())
}


// accept removes the oldest connection from the accept queue of a port.
func (st *stack) accept(port int) (key, bool) {
	l, ok := st.listeners[port]

	if !ok || len(l.queue) == 0 {
		return key{}, false
	}

	id := l.queue[0]
	l.queue = l.queue[1:]

	return id, true
}

// read removes up to n bytes from the receive buffer. It returns 0 bytes
// with ended set when the buffer is empty and a FIN has arrived.
func (s *socket) read(n int) (data string, ended bool) {
	n = min(n, len(s.received))
	data, s.received = string(s.received[:n]), s.received[n:]

	return data, n == 0 && s.finished
}


func main() {
	st := &stack{
		listeners:   map[int]*listener{80: {backlog: 2}},
		connections: map[key]*socket{},
		bufferSize:  8,
	}

	a, b, c := endpoint{"10.0.0.1", 5000}, endpoint{"10.0.0.2", 5000}, endpoint{"10.0.0.3", 5000}

	for _, seg := range []segment{{from: a, port: 80, syn: true}, {from: b, port: 80, syn: true},
		{from: c, port: 80, syn: true}, {from: c, port: 81, syn: true}} {
		fmt.Printf("SYN from %s to port %d: %s\n", seg.from, seg.port, st.receive(seg))
	}

	if id, ok := st.accept(80); ok {
		fmt.Printf("accept on port 80: the connection from %s\n", id.remote)
	}

	for _, from := range []endpoint{b, a, b, c} {
		fmt.Printf("5 bytes from %s to port 80: %s\n", from, st.receive(segment{from: from, port: 80, data: "hello"}))
	}

	conn, ok := st.connections[key{remote: b, port: 80}]

	if !ok {
		return
	}

	data, ended := conn.read(4)
	fmt.Printf("read of 4 bytes on the connection from %s: %q, window %d\n", b, data, conn.window())

	fmt.Printf("FIN from %s: %s\n", b, st.receive(segment{from: b, port: 80, fin: true}))

	data, ended = conn.read(16)
	fmt.Printf("read: %q, stream ended %v\n", data, ended)

	data, ended = conn.read(16)
	fmt.Printf("read: %q, stream ended %v\n", data, ended)
}

Listing 10.24 The complete sockmodel, in Rust. code/rust/sockmodel/src/main.rs

//! Models the receiving side of TCP in the kernel: a table of connections
//! keyed by addresses and ports, listeners with an accept queue of limited
//! length, and a receive buffer whose free space is the window offered to the
//! sender.
//!
//! Run it from code/rust with:
//!
//! ```text
//! cargo run -p sockmodel
//! ```

use std::collections::{HashMap, VecDeque};
use std::fmt;

/// An IP address and a port.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
struct Endpoint {
    address: &'static str,
    port: u16,
}

impl fmt::Display for Endpoint {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "{}:{}", self.address, self.port)
    }
}

/// One established connection: the bytes received and not yet read, the size
/// of the receive buffer, and whether the sender has finished.
struct Socket {
    received: VecDeque<u8>,
    capacity: usize,
    finished: bool,
}

impl Socket {
    /// The free space of the receive buffer: the number of bytes the sender
    /// may still send.
    fn window(&self) -> usize {
        self.capacity - self.received.len()
    }
}

/// Identifies a connection on this machine: the remote endpoint and the local
/// port. With the local address, which the model leaves out, it forms the 4
/// values that identify a TCP connection.
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
struct Key {
    remote: Endpoint,
    port: u16,
}

/// Accepts connections on a port. `queue` holds the connections that are
/// established and have not been returned by accept.
struct Listener {
    backlog: usize,
    queue: VecDeque<Key>,
}

/// The TCP state of a machine.
struct Stack {
    listeners: HashMap<u16, Listener>,
    connections: HashMap<Key, Socket>,
    buffer_size: usize,
}

/// A packet of TCP. `syn` opens a connection and `fin` ends the sender's
/// stream.
#[derive(Default)]
struct Segment {
    syn: bool,
    fin: bool,
    data: &'static str,
}

impl Stack {
    /// Handles a segment that arrives from `from` for local port `port`, and
    /// describes the result. A segment belongs to the connection with its
    /// remote endpoint and local port. Only when no such connection exists,
    /// and the segment is a SYN, is the listener on the port consulted.
    fn receive(&mut self, from: Endpoint, port: u16, segment: &Segment) -> String {
        let id = Key { remote: from, port };

        if let Some(socket) = self.connections.get_mut(&id) {
            return socket.deliver(segment);
        }

        if !segment.syn {
            return "reset, no connection".into();
        }

        let Some(listener) = self.listeners.get_mut(&port) else {
            return "reset, no listener on the port".into();
        };

        if listener.queue.len() == listener.backlog {
            return "dropped, the accept queue is full".into();
        }

        let socket = Socket {
            received: VecDeque::new(),
            capacity: self.buffer_size,
            finished: false,
        };

        self.connections.insert(id, socket);
        listener.queue.push_back(id);

        format!("established, accept queue {}", listener.queue.len())
    }
}

impl Socket {
    /// Puts the data of a segment into the receive buffer, up to the window,
    /// and records a FIN.
    fn deliver(&mut self, segment: &Segment) -> String {
        if segment.fin {
            self.finished = true;

            return "the sender's stream has ended".into();
        }

        let accepted = segment.data.len().min(self.window());
        self.received.extend(&segment.data.as_bytes()[..accepted]);

        format!("accepted {accepted}, window {}", self.window())
    }
}

impl Stack {
    /// Removes the oldest connection from the accept queue of a port.
    fn accept(&mut self, port: u16) -> Option<Key> {
        self.listeners.get_mut(&port)?.queue.pop_front()
    }
}

impl Socket {
    /// Removes up to `n` bytes from the receive buffer. It returns 0 bytes
    /// with the second value set when the buffer is empty and a FIN has
    /// arrived.
    fn read(&mut self, n: usize) -> (String, bool) {
        let n = n.min(self.received.len());
        let data: Vec<u8> = self.received.drain(..n).collect();

        (
            String::from_utf8_lossy(&data).into_owned(),
            n == 0 && self.finished,
        )
    }
}

fn endpoint(address: &'static str) -> Endpoint {
    Endpoint {
        address,
        port: 5000,
    }
}

fn main() {
    let listener = Listener {
        backlog: 2,
        queue: VecDeque::new(),
    };

    let mut stack = Stack {
        listeners: HashMap::from([(80, listener)]),
        connections: HashMap::new(),
        buffer_size: 8,
    };

    let (a, b, c) = (
        endpoint("10.0.0.1"),
        endpoint("10.0.0.2"),
        endpoint("10.0.0.3"),
    );

    let syn = Segment {
        syn: true,
        ..Segment::default()
    };

    for (from, port) in [(a, 80), (b, 80), (c, 80), (c, 81)] {
        let result = stack.receive(from, port, &syn);
        println!("SYN from {from} to port {port}: {result}");
    }

    if let Some(id) = stack.accept(80) {
        println!("accept on port 80: the connection from {}", id.remote);
    }

    let hello = Segment {
        data: "hello",
        ..Segment::default()
    };

    for from in [b, a, b, c] {
        let result = stack.receive(from, 80, &hello);
        println!("5 bytes from {from} to port 80: {result}");
    }

    let id = Key {
        remote: b,
        port: 80,
    };

    if let Some(socket) = stack.connections.get_mut(&id) {
        let (data, _) = socket.read(4);
        println!(
            "read of 4 bytes on the connection from {b}: {data:?}, window {}",
            socket.window()
        );
    }

    let fin = Segment {
        fin: true,
        ..Segment::default()
    };

    println!("FIN from {b}: {}", stack.receive(b, 80, &fin));

    if let Some(socket) = stack.connections.get_mut(&id) {
        for _ in 0..2 {
            let (data, ended) = socket.read(16);
            println!("read: {data:?}, stream ended {ended}");
        }
    }
}

References

• tcp(7), socket(7) and listen(2), Linux manual pages: https://man7.org/linux/man-pages/man7/tcp.7.html
• W. R. Stevens, B. Fenner, A. M. Rudoff, UNIX Network Programming, Volume 1, 3rd edition, Addison-Wesley, 2003.
• RFC 9293, Transmission Control Protocol (TCP): https://www.rfc-editor.org/rfc/rfc9293
• net/ipv4/tcp_input.c and net/ipv4/inet_hashtables.c, Linux 5.15: https://elixir.bootlin.com/linux/v5.15/source/net/ipv4/tcp_input.c

Namespaces

A namespace is a kernel object that gives a group of processes their own instance of a global resource: their own process numbers, their own host name, their own mounted file systems, or their own network interfaces. Processes inside a namespace see and change the instance of that namespace, and processes outside are unaffected. Namespaces are the isolation half of a container. Chapter 12 covers the other half, limits.

In this chapter, we will learn how a process enters a new namespace and what changes for it. We will first measure a child in new user, UTS and PID namespaces with nsfacts. Then we will write nsmodel, a model of nested PID namespaces and of the UTS namespace.

The kinds of namespace

Linux 5.15 has 8 kinds of namespace. Every process is in exactly 1 namespace of each kind, and a child starts in the namespaces of its parent. On a system without containers all processes share 1 namespace of each kind, and the resources appear global.

The kernel shows the namespaces of a process as symbolic links:

$ ls -l /proc/self/ns
cgroup -> cgroup:[4026531835]
ipc -> ipc:[4026531839]
mnt -> mnt:[4026531841]
net -> net:[4026531840]
pid -> pid:[4026531836]
pid_for_children -> pid:[4026531836]
time -> time:[4026531834]
user -> user:[4026531837]
uts -> uts:[4026531838]

The number in brackets identifies the namespace. Processes that show the same number for a kind are in the same namespace of that kind. pid_for_children is the PID namespace that children of the process will be created in, which can differ from the one the process itself is in.

The kernel’s structures

A task refers to its namespaces through struct nsproxy. This is the complete definition from include/linux/nsproxy.h of Linux 5.15, with comments added.

struct nsproxy {
        atomic_t count;                        /* tasks that share this nsproxy */
        struct uts_namespace *uts_ns;
        struct ipc_namespace *ipc_ns;
        struct mnt_namespace *mnt_ns;
        struct pid_namespace *pid_ns_for_children;
        struct net *net_ns;
        struct time_namespace *time_ns;
        struct time_namespace *time_ns_for_children;
        struct cgroup_namespace *cgroup_ns;
};

The structure is a set of pointers, 1 for each kind. Processes that are in the same namespaces share a single nsproxy. The user namespace is absent from the list: it is reached through the credentials of the task, task->cred->user_ns, because it decides what the task is permitted to do.

The PID namespace of a task is recorded in a different place, together with its numbers. These are the definitions from include/linux/pid.h and include/linux/pid_namespace.h.

struct upid {
        int nr;                                /* the number in 1 namespace */
        struct pid_namespace *ns;              /* that namespace */
};

struct pid {
        unsigned int level;                    /* depth of the task's PID namespace */
        struct upid numbers[1];                /* 1 entry per level, 0 to level */
        /* ... */
};

struct pid_namespace {
        struct task_struct *child_reaper;      /* its process 1, which adopts orphans */
        unsigned int level;                    /* 0 for the root namespace */
        struct pid_namespace *parent;          /* the enclosing namespace */
        struct user_namespace *user_ns;        /* the user namespace that owns it */
        /* ... */
};

Entering a namespace

A process gets into a namespace in 3 ways.

A namespace exists for as long as something refers to it: a process that is in it, or an open descriptor for its /proc link. When the last process leaves, the namespace is destroyed.

User namespaces

Creating a namespace requires a capability. Linux divides the power of the administrator into separate capabilities, and the one needed here is CAP_SYS_ADMIN. There is a single exception: any user may create a user namespace. Inside it, the kernel maps user numbers. The creator can map user 0 inside to its own user number outside, and it then holds all privileges over the resources that the user namespace owns, which include the other namespaces created together with it. Toward everything outside, it remains the unprivileged user it was. This is what lets an ordinary user start a container.

The mapping is written to /proc/<pid>/uid_map, a line of 3 numbers: the first user number inside, the first user number outside, and the count of numbers mapped. In the root namespace every number maps to itself:

$ cat /proc/self/uid_map
         0          0 4294967295

In the namespace that nsfacts creates below, the line is 0 1000 1: user 0 inside is user 1000 outside, and no other number is mapped.

The capabilities a process holds inside a user namespace are checked against the owner of the object it acts on. It may set the host name of a UTS namespace that its user namespace owns. It may not set the clock of the machine, mount a disk, or read a file that user 1000 cannot read, because those belong to the root user namespace.

Measuring namespaces

nsfacts uses golang.org/x/sys/unix in Go and the nix crate with the hostname, sched and user features in Rust.

$ cargo new nsfacts
$ cargo add -p nsfacts nix --features hostname,sched,user

The kernel shows the namespaces of a process as symbolic links under /proc/<pid>/ns. The target of a link, such as uts:[4026531838], names the namespace, and processes with equal targets are in the same namespace.

Listing 11.1 The imports of nsfacts, in Go. code/go/nsfacts/main.go

import (
	"bufio"
	"errors"
	"fmt"
	"log"
	"os"
	"os/exec"
	"strings"
	"syscall"

	"golang.org/x/sys/unix"
)

Listing 11.2 The imports of nsfacts, in Rust. code/rust/nsfacts/src/main.rs

use std::collections::HashMap;
use std::env;
use std::fs;
use std::io;
use std::process::{self, Command};

use nix::sched::{CloneFlags, unshare};
use nix::unistd::{getgid, gethostname, getpid, getuid, sethostname};

Listing 11.3 Reading the namespaces of a process, in Go. code/go/nsfacts/main.go

// kinds are the namespace types of Linux 5.15, as named under /proc/<pid>/ns.
var kinds = []string{"cgroup", "ipc", "mnt", "net", "pid", "time", "user", "uts"}

// namespaces returns the identity of each namespace of a process. The kernel
// shows it as the target of a symbolic link, such as "uts:[4026531838]".
// Processes are in the same namespace when the targets are equal.
func namespaces(pid string) (map[string]string, error) {
	ids := map[string]string{}

	for _, kind := range kinds {
		id, err := os.Readlink("/proc/" + pid + "/ns/" + kind)

		if err != nil {
			return nil, err
		}

		ids[kind] = id
	}

	return ids, nil
}

Listing 11.4 Reading the namespaces of a process, in Rust. code/rust/nsfacts/src/main.rs

/// The namespace types of Linux 5.15, as named under `/proc/<pid>/ns`.
const KINDS: [&str; 8] = ["cgroup", "ipc", "mnt", "net", "pid", "time", "user", "uts"];

/// Returns the identity of each namespace of a process. The kernel shows it
/// as the target of a symbolic link, such as `uts:[4026531838]`.
/// Processes are in the same namespace when the targets are equal.
fn namespaces(pid: &str) -> io::Result<HashMap<&'static str, String>> {
    KINDS
        .iter()
        .map(|&kind| {
            let id = fs::read_link(format!("/proc/{pid}/ns/{kind}"))?;

            Ok((kind, id.to_string_lossy().into_owned()))
        })
        .collect()
}

fn hostname() -> io::Result<String> {
    Ok(gethostname()?.to_string_lossy().into_owned())
}

The program runs itself again as a child with the argument inside. The child tries to set the host name to box and prints what it sees.

Listing 11.5 What the child reports, in Go. code/go/nsfacts/main.go

// name returns the symbolic name of a system call error, such as EPERM.
func name(err error) string {
	var errno unix.Errno

	if errors.As(err, &errno) {
		return unix.ErrnoName(errno)
	}

	return fmt.Sprint(err)
}

// inside runs in the child. It tries to change the host name, and prints
// what it sees as key=value lines for the parent to read.
func inside() error {
	outcome := "succeeds"

	if err := unix.Sethostname([]byte("box")); err != nil {
		outcome = "fails with " + name(err)
	}

	host, err := os.Hostname()

	if err != nil {
		return err
	}

	ids, err := namespaces("self")

	if err != nil {
		return err
	}

	uidMap, err := os.ReadFile("/proc/self/uid_map")

	if err != nil {
		return err
	}

	fmt.Printf("sethostname=%s\nhostname=%s\npid=%d\nuid=%d\n", outcome, host, os.Getpid(), os.Getuid())
	fmt.Printf("map=%s\nuts=%s\npidns=%s\n", strings.Join(strings.Fields(string(uidMap)), " "), ids["uts"], ids["pid"])

	return nil
}

Listing 11.6 What the child reports, in Rust. code/rust/nsfacts/src/main.rs

/// Runs in the child. It tries to change the host name, and prints what it
/// sees as key=value lines for the parent to read.
fn inside() -> io::Result<()> {
    let outcome = match sethostname("box") {
        Ok(()) => "succeeds".to_owned(),
        Err(error) => format!("fails with {error:?}"),
    };

    let ids = namespaces("self")?;
    let uid_map = fs::read_to_string("/proc/self/uid_map")?;
    let uid_map: Vec<&str> = uid_map.split_whitespace().collect();

    println!("sethostname={outcome}");
    println!("hostname={}", hostname()?);
    println!("pid={}", getpid());
    println!("uid={}", getuid());
    println!("map={}", uid_map.join(" "));
    println!("uts={}", ids["uts"]);
    println!("pidns={}", ids["pid"]);

    Ok(())
}

The child is started once in the parent’s namespaces and once in new user, UTS and PID namespaces. The languages create the namespaces differently. Go’s SysProcAttr passes the clone flags and the user mappings to the call that creates the child. unshare of a user namespace is refused in a process with several threads, which a Go program always is. The Rust program is single-threaded, so its helper process calls unshare, writes the mappings to /proc/self/uid_map and /proc/self/gid_map, and then starts the child.

Listing 11.7 Starting the child in new namespaces, in Go. code/go/nsfacts/main.go

// report runs this program again as a child and returns the lines the child
// prints, as a map. With isolate set, the child is created in new user, UTS
// and PID namespaces, and user 0 inside is mapped to the current user.
func report(isolate bool) (map[string]string, error) {
	child := exec.Command("/proc/self/exe", "inside")
	child.Stderr = os.Stderr

	if isolate {
		child.SysProcAttr = &syscall.SysProcAttr{
			Cloneflags:  syscall.CLONE_NEWUSER | syscall.CLONE_NEWUTS | syscall.CLONE_NEWPID,
			UidMappings: []syscall.SysProcIDMap{{ContainerID: 0, HostID: os.Getuid(), Size: 1}},
			GidMappings: []syscall.SysProcIDMap{{ContainerID: 0, HostID: os.Getgid(), Size: 1}},
		}
	}

	output, err := child.Output()

	if err != nil {
		return nil, err
	}

	seen := map[string]string{}

	for lines := bufio.NewScanner(strings.NewReader(string(output))); lines.Scan(); {
		if key, value, ok := strings.Cut(lines.Text(), "="); ok {
			seen[key] = value
		}
	}

	return seen, nil
}

Listing 11.8 Starting the child in new namespaces, in Rust. code/rust/nsfacts/src/main.rs

/// Moves this process into new user and UTS namespaces, maps user 0 inside to
/// the current user, and runs the program again as a child. A new PID
/// namespace applies to the children of the process that asks for it, so the
/// child is the first process of that namespace.
fn isolated() -> io::Result<()> {
    let (uid, gid) = (getuid(), getgid());

    unshare(CloneFlags::CLONE_NEWUSER | CloneFlags::CLONE_NEWUTS | CloneFlags::CLONE_NEWPID)?;

    fs::write("/proc/self/setgroups", "deny")?;
    fs::write("/proc/self/uid_map", format!("0 {uid} 1"))?;
    fs::write("/proc/self/gid_map", format!("0 {gid} 1"))?;

    let status = Command::new("/proc/self/exe").arg("inside").status()?;

    process::exit(if status.success() { 0 } else { 1 });
}

/// Runs this program again as a child and returns the lines the child prints,
/// as a map. With `isolate` set, the child is created in new namespaces.
fn report(isolate: bool) -> io::Result<HashMap<String, String>> {
    let mode = if isolate { "isolated" } else { "inside" };
    let output = Command::new("/proc/self/exe").arg(mode).output()?;

    let seen = String::from_utf8_lossy(&output.stdout)
        .lines()
        .filter_map(|line| line.split_once('='))
        .map(|(key, value)| (key.to_owned(), value.to_owned()))
        .collect();

    Ok(seen)
}

Listing 11.9 Comparing the child with the parent, in Go. code/go/nsfacts/main.go

func run() error {
	mine, err := namespaces("self")

	if err != nil {
		return err
	}

	name, err := os.Hostname()

	if err != nil {
		return err
	}

	for _, isolate := range []bool{false, true} {
		seen, err := report(isolate)

		if err != nil {
			return err
		}

		after, err := os.Hostname()

		if err != nil {
			return err
		}

		fmt.Printf("child in new user, UTS and PID namespaces: %v\n", isolate)
		fmt.Printf("  same UTS namespace as the parent %v, same PID namespace %v\n",
			seen["uts"] == mine["uts"], seen["pidns"] == mine["pid"])
		fmt.Printf("  sethostname %s; host name inside is \"box\" %v, outside unchanged %v\n",
			seen["sethostname"], seen["hostname"] == "box", after == name)
		fmt.Printf("  pid inside is 1 %v; uid inside is 0 %v\n", seen["pid"] == "1", seen["uid"] == "0")

		if fields := strings.Fields(seen["map"]); isolate && len(fields) == 3 {
			fmt.Printf("  uid map: user 0 inside is user %s outside\n", fields[1])
		}
	}

	return nil
}

Listing 11.10 Comparing the child with the parent, in Rust. code/rust/nsfacts/src/main.rs

fn run() -> io::Result<()> {
    let mine = namespaces("self")?;
    let name = hostname()?;

    for isolate in [false, true] {
        let seen = report(isolate)?;
        let get = |key: &str| seen.get(key).map_or("", String::as_str);

        println!("child in new user, UTS and PID namespaces: {isolate}");
        println!(
            "  same UTS namespace as the parent {}, same PID namespace {}",
            get("uts") == mine["uts"],
            get("pidns") == mine["pid"]
        );
        println!(
            "  sethostname {}; host name inside is \"box\" {}, outside unchanged {}",
            get("sethostname"),
            get("hostname") == "box",
            hostname()? == name
        );
        println!(
            "  pid inside is 1 {}; uid inside is 0 {}",
            get("pid") == "1",
            get("uid") == "0"
        );

        let map: Vec<&str> = get("map").split_whitespace().collect();

        if let (true, [_, outside, _]) = (isolate, map.as_slice()) {
            println!("  uid map: user 0 inside is user {outside} outside");
        }
    }

    Ok(())
}

child in new user, UTS and PID namespaces: false
  same UTS namespace as the parent true, same PID namespace true
  sethostname fails with EPERM; host name inside is "box" false, outside unchanged true
  pid inside is 1 false; uid inside is 0 false
child in new user, UTS and PID namespaces: true
  same UTS namespace as the parent false, same PID namespace false
  sethostname succeeds; host name inside is "box" true, outside unchanged true
  pid inside is 1 true; uid inside is 0 true
  uid map: user 0 inside is user 1000 outside

The measurement is clearer with the 2 points of view side by side. Nothing about the child process itself differs between the columns. What differs is the answer the kernel gives when it is asked.

PID namespaces

PID namespaces are nested. A new PID namespace is a child of the namespace in which it was created, and a process in it has a process number in its own namespace and in every namespace above it, up to the root. The numbers are independent of each other.

Visibility follows the nesting. A namespace sees the processes in itself and in the namespaces below it. It has no number for a process in a namespace above it or beside it, so it cannot name such a process in kill or find it under /proc.

The first process of a PID namespace takes the role that process 1 has on the whole machine: it adopts the orphans of its namespace, and when it ends, the kernel ends every other process in the namespace.

Modelling namespaces

The Go version links namespaces with pointers. The Rust version keeps them in vectors owned by a Kernel value and refers to them by index.

Listing 11.11 The imports of nsmodel, in Go. code/go/nsmodel/main.go

import "fmt"

Listing 11.12 PID namespaces, UTS namespaces and processes, in Go. code/go/nsmodel/main.go

// A pidNamespace hands out process numbers. level is its depth below the
// root namespace, which has level 0.
type pidNamespace struct {
	parent *pidNamespace
	level  int
	last   int
}

// allocate returns the next unused number of the namespace.
func (ns *pidNamespace) allocate() int {
	ns.last++

	return ns.last
}

// A utsNamespace holds the host name that its processes see.
type utsNamespace struct {
	hostname string
}

// A process belongs to one namespace of each kind. numbers holds its process
// number in every PID namespace from the root, at index 0, down to its own.
type process struct {
	pid     *pidNamespace
	uts     *utsNamespace
	numbers []int
}

Listing 11.13 PID namespaces, UTS namespaces and processes, in Rust. code/rust/nsmodel/src/main.rs

/// Hands out process numbers. `level` is its depth below the root namespace,
/// which has level 0. `parent` is the index of the enclosing namespace.
struct PidNamespace {
    parent: Option<usize>,
    level: usize,
    last: u32,
}

/// A process belongs to one namespace of each kind, named here by its index
/// in the kernel's tables. `numbers` holds its process number in every PID
/// namespace from the root, at index 0, down to its own.
struct Process {
    pid: usize,
    uts: usize,
    numbers: Vec<u32>,
}

/// The namespaces of a machine. A UTS namespace is the host name that its
/// processes see.
struct Kernel {
    pid_namespaces: Vec<PidNamespace>,
    hostnames: Vec<String>,
}

impl Kernel {
    /// Returns the next unused number of a PID namespace.
    fn allocate(&mut self, namespace: usize) -> u32 {
        self.pid_namespaces[namespace].last += 1;

        self.pid_namespaces[namespace].last
    }
}

fork walks from the child’s PID namespace up to the root and allocates a number in each. A new UTS namespace starts as a copy of the parent’s, so the child initially sees the same host name.

Listing 11.14 Creating a process, with or without new namespaces, in Go. code/go/nsmodel/main.go

// fork creates a child. By default the child is in its parent's namespaces.
// With newPID it is the first process of a new PID namespace below the
// parent's. With newUTS it gets a copy of the parent's UTS namespace.
func fork(parent *process, newPID, newUTS bool) *process {
	child := &process{pid: parent.pid, uts: parent.uts}

	if newPID {
		child.pid = &pidNamespace{parent: parent.pid, level: parent.pid.level + 1}
	}

	if newUTS {
		child.uts = &utsNamespace{hostname: parent.uts.hostname}
	}

	child.numbers = make([]int, child.pid.level+1)

	for ns := child.pid; ns != nil; ns = ns.parent {
		child.numbers[ns.level] = ns.allocate()
	}

	return child
}

Listing 11.15 Creating a process, with or without new namespaces, in Rust. code/rust/nsmodel/src/main.rs

impl Kernel {
    /// Creates a child. By default the child is in its parent's namespaces.
    /// With `new_pid` it is the first process of a new PID namespace below
    /// the parent's. With `new_uts` it gets a copy of the parent's UTS
    /// namespace.
    fn fork(&mut self, parent: &Process, new_pid: bool, new_uts: bool) -> Process {
        let (mut pid, mut uts) = (parent.pid, parent.uts);

        if new_pid {
            self.pid_namespaces.push(PidNamespace {
                parent: Some(parent.pid),
                level: self.pid_namespaces[parent.pid].level + 1,
                last: 0,
            });
            pid = self.pid_namespaces.len() - 1;
        }

        if new_uts {
            self.hostnames.push(self.hostnames[parent.uts].clone());
            uts = self.hostnames.len() - 1;
        }

        let mut numbers = vec![0; self.pid_namespaces[pid].level + 1];
        let mut namespace = Some(pid);

        while let Some(current) = namespace {
            numbers[self.pid_namespaces[current].level] = self.allocate(current);
            namespace = self.pid_namespaces[current].parent;
        }

        Process { pid, uts, numbers }
    }
}

numberIn decides visibility. It climbs from the process’s own namespace to the level of the asking namespace and checks that it arrives at that namespace.

Listing 11.16 The number of a process in a namespace, in Go. code/go/nsmodel/main.go

// numberIn returns the number by which a process is known in a namespace. A
// process is visible in its own PID namespace and in every namespace that
// encloses it, and in no other.
func (p *process) numberIn(ns *pidNamespace) (int, bool) {
	own := p.pid

	for own != nil && own.level > ns.level {
		own = own.parent
	}

	if own != ns {
		return 0, false
	}

	return p.numbers[ns.level], true
}

// seen describes a process as a namespace sees it.
func seen(p *process, ns *pidNamespace) string {
	if number, ok := p.numberIn(ns); ok {
		return fmt.Sprintf("pid %d", number)
	}

	return "not visible"
}

Listing 11.17 The number of a process in a namespace, in Rust. code/rust/nsmodel/src/main.rs

impl Kernel {
    /// Returns the number by which a process is known in a namespace. A
    /// process is visible in its own PID namespace and in every namespace
    /// that encloses it, and in no other.
    fn number_in(&self, process: &Process, namespace: usize) -> Option<u32> {
        let level = self.pid_namespaces[namespace].level;
        let mut own = Some(process.pid);

        while let Some(current) = own
            && self.pid_namespaces[current].level > level
        {
            own = self.pid_namespaces[current].parent;
        }

        if own != Some(namespace) {
            return None;
        }

        process.numbers.get(level).copied()
    }

    /// Describes a process as a namespace sees it.
    fn seen(&self, process: &Process, namespace: usize) -> String {
        match self.number_in(process, namespace) {
            Some(number) => format!("pid {number}"),
            None => "not visible".to_owned(),
        }
    }
}

Listing 11.18 The trace, in Go. code/go/nsmodel/main.go

func main() {
	root := &pidNamespace{}
	host := &utsNamespace{hostname: "host"}

	first := &process{pid: root, uts: host, numbers: []int{root.allocate()}}
	shell := fork(first, false, false)
	a := fork(shell, true, true)
	child := fork(a, false, false)
	b := fork(shell, true, true)
	nested := fork(child, true, false)

	rows := []struct {
		name string
		p    *process
	}{{"first process", first}, {"shell", shell}, {"first process of A", a}, {"its child", child},
		{"first process of B", b}, {"nested below A", nested}}

	for _, row := range rows {
		fmt.Printf("%-20s numbers from the root namespace down: %v\n", row.name, row.p.numbers)
	}

	fmt.Printf("the first process of B, seen from the root namespace: %s\n", seen(b, root))
	fmt.Printf("the first process of B, seen from A: %s\n", seen(b, a.pid))
	fmt.Printf("the shell, seen from A: %s\n", seen(shell, a.pid))
	fmt.Printf("the nested process, seen from A: %s\n", seen(nested, a.pid))

	a.uts.hostname = "box"
	fmt.Printf("host name set to \"box\" in A: A sees %q, its child %q, B %q, the shell %q\n",
		a.uts.hostname, child.uts.hostname, b.uts.hostname, shell.uts.hostname)
}

Listing 11.19 The trace, in Rust. code/rust/nsmodel/src/main.rs

/// Formats numbers the way Go prints a slice.
fn list(numbers: &[u32]) -> String {
    let words: Vec<String> = numbers.iter().map(u32::to_string).collect();

    format!("[{}]", words.join(" "))
}

fn main() {
    let root = PidNamespace {
        parent: None,
        level: 0,
        last: 0,
    };

    let mut kernel = Kernel {
        pid_namespaces: vec![root],
        hostnames: vec!["host".to_owned()],
    };

    let first = Process {
        pid: 0,
        uts: 0,
        numbers: vec![kernel.allocate(0)],
    };

    let shell = kernel.fork(&first, false, false);
    let a = kernel.fork(&shell, true, true);
    let child = kernel.fork(&a, false, false);
    let b = kernel.fork(&shell, true, true);
    let nested = kernel.fork(&child, true, false);

    let rows = [
        ("first process", &first),
        ("shell", &shell),
        ("first process of A", &a),
        ("its child", &child),
        ("first process of B", &b),
        ("nested below A", &nested),
    ];

    for (name, process) in rows {
        println!(
            "{name:<20} numbers from the root namespace down: {}",
            list(&process.numbers)
        );
    }

    println!(
        "the first process of B, seen from the root namespace: {}",
        kernel.seen(&b, 0)
    );
    println!(
        "the first process of B, seen from A: {}",
        kernel.seen(&b, a.pid)
    );
    println!("the shell, seen from A: {}", kernel.seen(&shell, a.pid));
    println!(
        "the nested process, seen from A: {}",
        kernel.seen(&nested, a.pid)
    );

    kernel.hostnames[a.uts] = "box".to_owned();
    println!(
        "host name set to \"box\" in A: A sees {:?}, its child {:?}, B {:?}, the shell {:?}",
        kernel.hostnames[a.uts],
        kernel.hostnames[child.uts],
        kernel.hostnames[b.uts],
        kernel.hostnames[shell.uts]
    );
}

first process        numbers from the root namespace down: [1]
shell                numbers from the root namespace down: [2]
first process of A   numbers from the root namespace down: [3 1]
its child            numbers from the root namespace down: [4 2]
first process of B   numbers from the root namespace down: [5 1]
nested below A       numbers from the root namespace down: [6 3 1]
the first process of B, seen from the root namespace: pid 5
the first process of B, seen from A: not visible
the shell, seen from A: not visible
the nested process, seen from A: pid 3
host name set to "box" in A: A sees "box", its child "box", B "host", the shell "host"

Inspecting namespaces on a live system