Foreword

The most important change in Linux networking in the last decade did not happen in a new protocol, a new syscall, or a new NIC feature. It happened in a tiny register machine that lives inside the kernel, has no loops you can abuse, and is allowed to touch kernel memory only through a list of approved functions. That machine is the extended Berkeley Packet Filter - eBPF - and it turned the kernel from a black box into a programmable platform.

Before eBPF, the options for a network engineer were stark. You could ship packets through the kernel’s fixed pipeline and hope the performance was good enough. You could reach for kernel modules, and take on kernel crash bugs, out-of-tree drift, and a maintainer community that rightly treats every new module as a liability. Or you could bypass the kernel entirely with DPDK and friends, and give up the kernel’s protection model, its TCP stack, and its tooling. eBPF broke that trilemma: programs that run inside the kernel at packet rate, that the verifier proves safe before they ever execute, and that you can load, update, and unload from userspace without a reboot.

The second half of the story is Rust. eBPF is a small, memory-unsafe C-like language with a strict verifier; the discipline the verifier demands - no unbounded loops, no out-of-bounds access, no leaked references - is exactly the discipline Rust’s type system enforces at compile time. The Aya project lets you write eBPF programs in Rust, compile them with a modified LLVM back end, and load them with a safe, idiomatic Rust userspace library. For the first time, the whole stack - the program that runs in the kernel and the userspace control plane that feeds it maps - can be written in one language, by one engineer, in one repository.

Why This Book Exists

There are excellent references on eBPF, and excellent references on Rust, and excellent references on Kubernetes networking. This book is about the place where all three meet: the kernel data plane of modern cloud infrastructure. Cilium - the most successful eBPF project in production - replaced iptables-based kube-proxy with eBPF load balancers, replaced userspace proxies with in-kernel socket redirects, and built a service mesh whose data path is a set of eBPF programs written (in increasing amounts) in Rust. Facebook, Google, Netflix, and the major clouds run eBPF in their routers, their DDoS filters, their security agents, and their container runtimes. If you write software that moves packets, eBPF is your platform.

Facts alone will not get you there. This book builds a model: the packet path from the NIC to your socket, the eBPF virtual machine and its verifier, the map contract between kernel and userspace, the socket layer, the Kubernetes service model, and the Cilium architecture - each layer defined before it is used, each claim accompanied by reasoning, each line of code commented, each idea drawn. The same discipline the C++ systems book applies to memory, this book applies to packets.

What You Will Build

Every chapter builds toward one project: a service-mesh data path in Rust with eBPF. The capstone wires together the pieces the earlier chapters taught you to trust: an XDP load balancer that steers packets by a hash, sockmap programs that redirect connections between pod replicas without touching userspace, and a Hubble-style observability pipeline that ships packet events out of the kernel through a BPF ring buffer. You will run it in a local kind cluster with Cilium installed, watch the same packets from two viewpoints - kernel and userspace - and finish with a system you could extend into a production load balancer or an ingress controller.

You will not build a toy. You will build the same structures the cloud providers ship, using the same disciplines: verifier-friendly program shapes, map layouts chosen for the access pattern, error paths that are logged not swallowed, and reproducible benchmarks.

Who This Book Is For

You should read this book if:

• You are a software engineer working at the networking layer - building load balancers, proxies, gateways, security agents, or anything that touches packets or sockets - and you want to move from “it goes through the kernel somehow” to a first-principles model.
• You know Rust and want to write systems programs that run inside the kernel, and you want to do it without C and without kernel modules.
• You run or build Kubernetes networking - CNIs, service meshes, network policies - and you want to understand what Cilium actually does in the data path instead of treating it as a black box.
• You are preparing for a systems or networking interview at a cloud provider, a networking company, or anywhere the questions involve packet paths, sockets, and kernel data structures.
• You ship software whose performance budget is measured in packets per second and whose correctness budget is zero.

You do not need to have written eBPF before, but you do need to be willing to sit with the verifier and with the Linux source tree. This book does not hand-wave either one. Every term is defined when it first appears; every helper is described before it is used; every number comes with the reasoning behind it.

The Structure

The book is organised into six parts:

Part I - Foundations (Chapters 1-3) builds the model: the Linux packet path from the NIC to your socket, the eBPF virtual machine and its safety model, and the Aya toolchain that brings it all to Rust. Everything else is an application of these three chapters.

Part II - eBPF Programs, Maps & the Data Path (Chapters 4-6) is the hands-on core: the map contract between kernel and userspace, the XDP hook at line rate, and the traffic control hooks on the other side of the stack.

Part III - Protocols, Sockets & the Kernel Data Plane (Chapters 7-9) goes deep on the protocol layer: TCP/IP and the receive path, sockmap and SK_MSG redirect in the socket layer, and cgroup hooks for filtering and resource control.

Part IV - Kubernetes Networking & Cilium (Chapters 10-12) moves to the cloud: the Kubernetes networking model and CNI, Cilium’s eBPF data plane that replaces kube-proxy, and the service mesh, security policies, and Hubble observability built on top.

Part V - Production Systems Engineering (Chapters 13-15) covers what it takes to ship: performance engineering and profiling, testing and debugging against the verifier, and the security model - and attack surface - of eBPF itself.

Part VI - The Capstone (Chapter 16) closes the loop: a complete service-mesh data path in Rust, running against a real cluster, using every tool from the previous five parts.

A Note on Platforms

eBPF programs run in the Linux kernel, so the code in this book requires Linux with a recent kernel (5.15+ for everything we use; 6.x preferred) and CONFIG_BPF, CONFIG_BPF_SYSCALL, CONFIG_DEBUG_INFO_BTF, and the XDP and sockmap subsystems enabled. Everything else - the Aya userspace side, the maps, the tests - is ordinary Rust. If you are on macOS or Windows, the easiest path is a Linux VM or a cloud instance; the book tells you what to check and how to verify each prerequisite. The capstone additionally needs kind or a small Kubernetes cluster.

The machine this book assumes as its running example is a modern Linux x86-64 server (a 2-3 GHz Xeon or EPYC, 64-byte cache lines, a 25-100 Gb/s NIC with RSS) running a 6.x kernel. The numbers differ between generations, but the structure - and the reasoning - does not.

Welcome. Let’s build the model, and then let’s build the data path.

Chapter 1: The Linux Networking Stack from First Principles

“A packet is born in the NIC and dies in your socket. Everything in between - every ring, every queue, every copy, every syscall - is a decision someone made about speed versus complexity.”

Every eBPF program in this book does one of two things: it observes the packet path or it changes it. You cannot do either well without a precise model of that path. This chapter builds the model from the wire to the socket: the NIC’s DMA rings, the interrupt-versus-poll trade, the softirq where the stack lives, the protocol dispatch, and the socket receive queue where your recvfrom finds its data. Along the way we mark the exact points where eBPF can attach - because every hook in this book is a point on this path, and the cost of a hook is the cost of the path it sits on.

1.1 The NIC and the Ring

A modern NIC does not push packets at the CPU. It pulls - or rather, the driver and the NIC cooperate on a shared structure called a ring buffer (also known as a descriptor ring, or in the modern API, a page pool). The driver allocates a region of memory, fills it with empty packet buffers (pages), and tells the NIC, “here is where you may write incoming frames”. The NIC DMAs a frame into the next free buffer and advances its head pointer. The driver’s job is to keep ahead of the NIC: reclaim consumed buffers and replenish empty ones before the ring underruns.

Two facts follow immediately, and both shape everything else in this book:

1. There is no syscall on the receive path. The bytes travel from the wire into kernel memory by DMA. The kernel does not ask for them; the NIC deposits them. This is why eBPF can run at line rate: the packets are already in memory the kernel owns.
2. The ring is a producer-consumer queue in memory. The same mental model - head, tail, slots, one side writes, one side reads - appears again as the BPF ring buffer (Chapter 4) and as sockmap’s socket queues (Chapter 8). Learning to think in rings is the single most transferable skill in this book.

1.2 Interrupts vs. NAPI: Why the Kernel Polls

When a frame arrives, the NIC raises a hardware interrupt to get the CPU’s attention. Interrupts are how the NIC says “work is available”; they are also expensive - each one costs thousands of cycles of register save, pipeline drain, and return. Under heavy load, interrupt storms used to melt the machine: the CPU spent all its time servicing interrupts and none processing packets (the classic receive livelock).

The fix is NAPI (New API), the polled receive path introduced in Linux 2.4 and universal since. The driver first receives packets in interrupt mode, but on the first interrupt it does two things: it disables further interrupts and it schedules a softirq poll on a per-CPU napi_struct. The poll loop then drains the ring in batches - dozens or hundreds of frames per poll - and only re-enables interrupts when the ring is empty. The result: at low load, interrupts are rare and cheap; at high load, the CPU polls in efficient batches and the interrupt rate drops to near zero.

This is the first cost you must internalise: the CPU that runs the poll loop is the CPU that runs the whole stack for that queue. Which CPU is determined by RSS (Receive Side Scaling): the NIC hashes each frame’s headers and steers it to one of several queues, each with its own IRQ and its own CPU. Hash the (src ip, dst ip, src port, dst port) tuple and all packets of one flow land on the same CPU - which preserves ordering and spreads flows across cores. Chapter 7 returns to RSS with the details that matter for eBPF.

1.3 softirq: Where the Stack Actually Runs

The poll loop runs in softirq context - a deferred interrupt context that preempts whatever the CPU was doing. The stack, from netif_receive_skb onward, runs inside this softirq, not in the context of any process. Three consequences matter for everything we build:

• The receive path is not your process. When a packet arrives for your socket, the kernel does the protocol work right then, in softirq context, on the RSS-assigned CPU - not when you happen to call recv.
• Latency is budget. Every step between the ring and your socket costs cycles inside a context you do not control. The kernel’s engineering objective is to make those steps cheap: merge packets (GRO), delay allocation, batch work.
• eBPF hooks in the path inherit that budget. An XDP program (Chapter 5) runs in the driver’s poll context before the skb is even allocated. A tc ingress program runs after GRO, on the skb. Both have the same implicit deadline: finish before the next poll batch, or you become the bottleneck.

1.4 GRO and the Cost of Bigger Packets

The stack’s most effective optimisation is not cleverer parsing; it is fewer, bigger packets. GRO (Generic Receive Offload) merges consecutive TCP segments that arrive back-to-back for the same flow into one larger skb before the stack processes them. Where the NIC would hand you ten 1448-byte segments, the stack hands the TCP layer one ~14 KB super-packet (up to the MTU-limited gro_max_size). One skb means one round of header parsing, one trip through netfilter and the protocol layers, one entry into the socket queue instead of ten. The bytes are identical; the work per byte collapses.

GRO exists because of its transmit twin, GSO (Generic Segmentation Offload): the kernel hands the NIC one large skb, and the NIC segments it into wire-sized frames. GRO reverses that at receive. Whenever you read a Cilium or kernel BPF program that seems to assume “one skb per TCP segment”, remember GRO: by the time tc sees the packet, it may represent a run of segments. Chapter 7 covers the precise implications.

1.5 Protocol Dispatch and the Path to the Socket

After GRO, netif_receive_skb delivers the skb to protocol handlers: ptype_all (for taps like tcpdump and AF_PACKET sockets) and ptype_base (the IP, ARP, and other dispatch tables keyed by ethertype). The IP layer runs, netfilter’s hooks fire (PREROUTING, conntrack, FORWARD/INPUT), and for a locally destined packet the transport layer takes over. TCP validates the segment, updates the receive window, and queues the data into the socket’s receive buffer - an sk_buff_head protected by the socket lock - then wakes a blocked reader if one exists.

The receive queue is where the kernel’s job ends and your program’s begins:

recvfrom(fd, buf, n, 0);   // syscall
  -> copy data from socket receive queue into buf
  -> release the skb back to the page pool

That copy - from kernel buffer to userspace buffer - is the last copy in the receive path, and for most workloads it is the only one that touches your program. This is the default kernel path: interrupt, DMA, poll, GRO, stack, queue, syscall, copy. Roughly 2-10 us end-to-end with jitter, and a full syscall round trip per recvfrom. Chapter 12 of the C++ companion book (“Low-Latency I/O”) showed how mmap, io_uring, and kernel bypass shave this path; this book’s eBPF chapters show how to inhabit it.

1.6 Where eBPF Hooks In: The Map of This Book

The path you just built has a hook at almost every stage, and each hook is a chapter:

The ordering principle: the earlier the hook, the fewer kernel allocations and copies it has already paid for, and the fewer protocol guarantees it can rely on. XDP sees a raw Ethernet frame with no skb, no routing decision, no connection tracking - and can drop it in ~100 ns. A socket filter sees the fully validated, queued data of an established connection - and cannot avoid the stack’s cost, because the stack already ran.

1.7 The Cost Model You Will Quote

For the rest of the book, these are the numbers to reason with (details in Appendix B):

• Per-packet budget: at R Mpps on a 4 GHz core, you have 4000 / R cycles per packet. At 1 Mpps you have 4000 cycles - plenty for a hash lookup and a redirect. At 10 Mpps you have 400 - enough for a well-shaped XDP program, nothing more. Every eBPF design question in this book is, at bottom, a question about this budget.
• The two taxes: copies and syscalls. The default path pays both (one memcpy, one syscall per recv). eBPF’s value proposition is that it can run before either tax applies - or eliminate them entirely (sockmap, Chapter 8).
• The hook placement rule: a program attached later in the path sees more (validated, merged, routed) data and costs more to skip. Choose the hook that sees just enough to make your decision.

Source: code/ch01_packet_path/parse_frame.rs

// code/ch01_packet_path/parse_frame.rs
// Chapter 1 demo - walk an Ethernet frame with bounds-checked slice
// helpers. Portable: compiles and runs anywhere (no kernel, no BPF).
// Build: rustc parse_frame.rs && ./parse_frame
use std::net::Ipv4Addr;

const ETH_HDR_LEN: usize = 14;  // dst MAC 6 + src MAC 6 + ethertype 2
const IP_HDR_LEN: usize = 20;   // IPv4 without options
const ETH_P_IP: u16 = 0x0800;   // IPv4 ethertype (big-endian on the wire)
const PROTO_UDP: u8 = 17;

// The same bounds-checked readers the kernel-side programs use (net.rs).
fn u16_be(f: &[u8], off: usize) -> Option<u16> {
    let s = f.get(off..off + 2)?;
    Some(u16::from_be_bytes([s[0], s[1]]))
}
fn u32_be(f: &[u8], off: usize) -> Option<u32> {
    let s = f.get(off..off + 4)?;
    Some(u32::from_be_bytes([s[0], s[1], s[2], s[3]]))
}

/// Parse an IPv4 frame into (src, dst, protocol), or None if it is not
/// an IPv4 frame or is truncated. Pure and total - never panics.
fn parse_ipv4(frame: &[u8]) -> Option<(Ipv4Addr, Ipv4Addr, u8)> {
    // ethertype at offset 12; anything that is not IPv4 is not ours.
    if u16_be(frame, 12)? != ETH_P_IP {
        return None;
    }
    let proto = frame.get(ETH_HDR_LEN + 9)?;
    let src = Ipv4Addr::from(u32_be(frame, ETH_HDR_LEN + 12)?);
    let dst = Ipv4Addr::from(u32_be(frame, ETH_HDR_LEN + 16)?);
    Some((src, dst, *proto))
}

fn main() {
    // Build a synthetic 60-byte frame: eth header + IPv4 header + payload.
    let mut frame = vec![0u8; ETH_HDR_LEN + IP_HDR_LEN + 20];
    frame[12..14].copy_from_slice(&ETH_P_IP.to_be_bytes());
    frame[ETH_HDR_LEN + 0] = 0x45;                  // IPv4, IHL = 5 words
    frame[ETH_HDR_LEN + 8] = 64;                    // TTL
    frame[ETH_HDR_LEN + 9] = PROTO_UDP;
    frame[ETH_HDR_LEN + 12..ETH_HDR_LEN + 16].copy_from_slice(&[192, 168, 1, 10]);
    frame[ETH_HDR_LEN + 16..ETH_HDR_LEN + 20].copy_from_slice(&[10, 0, 0, 1]);

    match parse_ipv4(&frame) {
        Some((src, dst, proto)) => {
            println!("ethertype = 0x{ETH_P_IP:04x} (IPv4)");
            println!("protocol = {proto}, src = {src}, dst = {dst}");
            println!("packet parse complete; no syscall, no copy, just a header walk");
        }
        None => println!("not an IPv4 frame (or truncated)"),
    }
}

The demo is deliberately boring: it is a header walk with explicit bounds checks, which is exactly the shape of a verifier-accepted XDP program, minus the kernel. If you can explain why every assert! here is mandatory, you already understand the single most important constraint in this book: eBPF programs never trust the packet; they check every access against data_end.

Hands-On Lab

# 1. Build and run the portable header-walk demo.
rustc code/ch01_packet_path/parse_frame.rs && ./parse_frame

# 2. See the real thing: capture a frame and compare the offsets you
#    parsed with what the wire actually carries.
sudo tcpdump -i any -c 1 -nn -XX    # the -XX shows raw bytes, hex + ascii

Walk the hexdump: bytes 0-5 dst MAC, 6-11 src MAC, 12-13 ethertype (08 00 = IPv4), 14-33 IPv4 header, and find the protocol byte at 14+9. If you can point at each field, you already know how an XDP program parses (Chapter 5 does the same walk in the kernel).

Summary

• The receive path is DMA into a ring, then a poll: the NIC writes frames into kernel memory; NAPI disables interrupts under load and polls the ring in batches inside softirq context.
• RSS decides which CPU owns a queue by hashing the flow tuple; the CPU that polls is the CPU that runs the whole stack for that flow.
• GRO merges back-to-back TCP segments into one skb; fewer, bigger packets mean fewer rounds of parsing and queueing.
• The path ends with a socket receive queue: the kernel queues data, then recvfrom copies it out. One syscall, one copy - the two taxes.
• eBPF hooks exist at almost every stage: earlier hooks (XDP) see raw frames before skb allocation; later hooks (socket, sockmap, cgroup) see validated connection state. Earlier is cheaper and less informed.
• The whole book is a budget question: 4000 / R cycles per packet at R Mpps on a 4 GHz core. Design programs inside that budget.

Next: Chapter 2 opens the box - the eBPF virtual machine, the verifier that makes in-kernel programs safe, and the JIT that makes them fast.

Chapter 2: The eBPF Virtual Machine - Verifier, JIT & the Safety Model

“eBPF is not ‘safe because it is simple’. It is safe because a proof checker refuses to load programs it cannot prove safe. The verifier is the product.”

Chapter 1 showed where eBPF hooks live. This chapter opens the box: what an eBPF program actually is, how the kernel proves it safe before it ever runs, how the JIT turns it into native code, and why all of this is the reason you can write kernel code without kernel modules. It is also the chapter where most newcomers get their first rejection - and learning to read a verifier rejection like a compiler error is a skill this book practices from Chapter 3 onward.

2.1 The eBPF Instruction Set: A Register Machine

An eBPF program is a sequence of instructions for a small, formally defined virtual machine. The classic description: 11 general-purpose 64-bit registers (r0 through r10), a program counter, a small set of instruction classes (ALU, jumps, memory, and the two special classes - calls to helpers and returns to the kernel), and no arbitrary memory access: the only memory an eBPF program may touch is its stack, the context passed in by the hook, and the maps it declares. Everything else must be reached through a helper.

The instruction encoding mirrors that of classic BPF (the packet filter language of tcpdump), extended to 64 bits:

Programs are limited in size (BPF_MAXINSNS, one million instructions on 6.x kernels) and, for most program types, may not contain unbounded loops - though bounded loops with an explicit upper bound have been accepted since Linux 5.3. The important consequence for you as a Rust programmer: the verifier must be able to prove that every loop terminates, every memory access is in-bounds, and every pointer stays valid - and it checks this statically, before the program runs.

2.2 Helpers: The Kernel’s Whitelist

An eBPF program cannot call arbitrary kernel functions. It can call only helpers: a curated, versioned list of kernel functions exposed to BPF, each one implemented in kernel/bpf/helpers.c and friends. The helper list is the kernel’s attack surface reduction: instead of giving programs the ability to call anything, the kernel gives them a fixed vocabulary - bpf_map_* for map access, bpf_skb_* and bpf_redirect* for networking, bpf_get_current_* for task introspection, bpf_ktime_get_ns for time - and proves at load time that every call instruction names a valid helper with the right signature.

For Rust users this is a moment of recognition: helpers are the kernel’s trait object for eBPF - a fixed, checked interface between the untrusted program and the trusted kernel. Aya models them as functions in aya_bpf::helpers, each with the exact signature the kernel expects; you call bpf_redirect() the same way you would call any other function, and the verifier checks the arguments at load time.

2.3 Maps: The Only Shared State

Between the kernel side and the userspace side of an eBPF program sits a map: a kernel-allocated data structure with a fixed key/value type, created by the userspace loader and referenced by the program through a file descriptor. Maps are the only state an eBPF program can share - with other programs, with the kernel, and with userspace. The map is also the only persistent thing: when the program that uses a map exits, the map can outlive it, which is how userspace reads counters long after a hook fired.

The map types that matter in this book (full treatment in Chapter 4):

Two map rules govern every design in this book: choose the map for the access pattern (a hot counter is PERCPU_ARRAY, not HASH), and define the key/value layout once, in one place, shared by kernel and userspace - in Rust, one #[repr(C)] struct in a shared module.

2.4 The Verifier: A Proof Checker in the Load Path

When userspace issues bpf(BPF_PROG_LOAD, ...), the kernel does not run the program. It runs the verifier: a static analyser that walks the instruction stream and builds a symbolic execution of the program with a per-register type and value range. The verifier’s checks, roughly in order of fame:

1. Type safety. Every register has a type: scalar, packet pointer, map value pointer, stack pointer, context pointer. Operations are checked against types; a scalar cannot be used as a pointer, and a packet pointer cannot be compared with a map pointer.
2. Bounds. Packet access is allowed only where the verifier can prove ptr + offset <= data_end. Map keys and values are bounds-checked against the map definition. Stack access is checked against the frame.
3. Termination. Loops must have a provable bound; recursion is forbidden entirely (the verifier rejects a program that could call itself).
4. No leaks / no dangling. Pointers that escape the program (e.g. into a map value) must be accompanied by the correct reference count; the verifier tracks references and rejects leaks.
5. Helper argument validation. Every helper call is checked against the helper’s declared argument types.

The verifier is a static checker: it does not run your program with test data, it proves properties over all possible executions. When it fails, it emits a rejection log - the invalid access to packet, off=42 size=4, R4=... messages that Chapter 14 teaches you to read like compiler errors.

# A real rejection (simplified), from bpftool prog load:
# libbpf: prog 'xdp_parse': BPF program load failed: Permission denied
# R2=pkt(id=0,off=14,r=14,imm=0) R2_w=pkt(off=14,r=14,imm=0) R10=fp0
# 12: (71) r1 = *(u8 *)(r2 + 20)     ; access beyond data_end
# invalid access to packet, off=20 size=1, R2 has 14 bytes of readable data

Read that message carefully: off=14 is where the packet pointer was, r=14 says only 14 bytes (the Ethernet header) are known readable, and the program tried to read byte 20 without first checking data_end. The fix - the one Chapter 5 makes second nature - is the explicit bounds check before every header field access.

2.5 The JIT: From Verification to Native Code

Verification is the price of safety; the JIT is the refund. After the verifier accepts a program, the kernel’s BPF JIT compiler translates the eBPF instructions into native instructions for the host architecture (x86-64, arm64, riscv, s390, and others). The JIT output runs with the same privileges as kernel code - which is precisely why verification is non-negotiable - and at the same speed as hand-written kernel code. For network programs the JIT output is what executes per packet; a well-shaped XDP program is, after JIT, a few dozen native instructions.

The two-step pipeline - verify, then JIT - is the architectural answer to a question that predates eBPF: how do you get kernel performance without kernel risk? Modules answer “trust the author”. eBPF answers “prove the program, then let it run”. For the systems engineer, the practical consequence is a new failure mode: a program can be syntactically valid Rust, semantically what you meant, and still rejected at load time because the verifier cannot prove a bound. Chapter 14 is devoted to this class of bug.

2.6 BTF and CO-RE: Programs That Survive Kernel Upgrades

There is one more piece of the model you will meet constantly: BTF (BPF Type Format), a compact type graph the kernel can emit for its own data structures, and CO-RE (Compile Once, Run Everywhere), the relocation system that lets one compiled program adapt to different kernels at load time. The problem: an eBPF program that reads struct task_struct directly would break when the kernel changes the struct. With CO-RE, the compiled program records which fields it needs as relocations; the loader rewrites the offsets against the running kernel’s BTF before the verifier sees the program. Aya and bpf-linker handle CO-RE relocations for you; the practical effect is that your Rust eBPF programs do not need to be recompiled for each kernel - only BTF-enabled.

BTF also gives you a debugging superpower: bpftool btf dump and pahole let you inspect kernel struct layouts precisely, which is how you write correct kprobe programs (Chapter 7) without guessing field offsets.

2.7 The Lifecycle: Load, Attach, Run, Unload

Put the pieces together and the lifecycle of an eBPF program is:

1. Compile (Chapter 3): Rust kernel-side code is compiled with the bpfel-unknown-none target and bpf-linker into a single ELF object containing the programs and their map definitions.
2. Load: the userspace side (Aya’s Ebpf::load) reads that ELF, creates the maps, and issues BPF_PROG_LOAD for each program. The verifier runs now - this is where rejections happen.
3. Attach: the program is attached to its hook (an interface for XDP, a qdisc for tc, a cgroup, a function for kprobes).
4. Run: the kernel invokes the program per event - per packet, per syscall, per event. It may read and write maps, and it returns an action the kernel obeys (XDP_DROP, TC_ACT_OK, …).
5. Unload: when the userspace process exits (or an explicit detach runs), the links are torn down and the programs and maps are freed.

Everything after step 3 is per packet or per event; everything before is control plane. Keeping those two planes separate - no map updates on the hot path, no loading on the hot path - is the discipline Chapter 13 turns into a performance methodology.

Source: code/ch02_ebpf_vm/verifier_notes.rs

// code/ch02_ebpf_vm/verifier_notes.rs
// Chapter 2 demo - the verifier's mental model, in Rust shapes.
// Portable: compiles and runs anywhere.
// Build: rustc verifier_notes.rs && ./verifier_notes
#![allow(dead_code)] // the demo only constructs Packet; keep the model complete

#[derive(Clone, Copy, Debug, PartialEq)]
enum RegType {
    Scalar,              // a plain number, no memory behind it
    Packet(u64),         // pointer into packet memory + known readable bytes
    MapValue,            // pointer into a map value
    Stack,               // pointer into the program's own stack frame
    Context,             // the hook-provided context pointer
}

struct Reg {
    ty: RegType,
    val: i64,   // known value (or the range bound the verifier tracks)
}

/// The check the verifier applies to a memory access: the requested extent
/// must fit inside the pointer's known-readable region.
fn check_read(reg: Reg, offset: u64, size: u64) -> Result<(), String> {
    match reg.ty {
        // A scalar has no memory: using it as a pointer is a type error.
        RegType::Scalar => Err("scalar register cannot be dereferenced".into()),
        // Packet pointers carry a 'readable bytes' count (r=... in logs).
        RegType::Packet(readable) => {
            let end = offset.checked_add(size).ok_or("offset overflow")?;
            if end <= readable {
                Ok(())
            } else {
                Err(format!("invalid access to packet, off={offset} size={size}, only {readable} bytes readable"))
            }
        }
        // Map/stack/context pointers are checked against their own sizes
        // at load time; nothing to prove here for the demo.
        RegType::MapValue | RegType::Stack | RegType::Context => Ok(()),
    }
}

fn main() {
    // The exact rejection from Section 2.4: byte 20 read when only the
    // 14-byte Ethernet header is known readable.
    let eth_only = Reg { ty: RegType::Packet(14), val: 0 };
    let rejection = check_read(eth_only, 20, 1).unwrap_err();
    println!("verifier: {rejection}");

    // The fix: prove data_end - data >= 24 BEFORE reading off=20 size=4.
    let checked = Reg { ty: RegType::Packet(24), val: 0 };
    match check_read(checked, 20, 4) {
        Ok(()) => println!("accepted after bounds check (r=24, off=20, size=4)"),
        Err(e) => println!("verifier: {e}"),
    }
}

The demo compiles and runs anywhere - the verifier it models only exists in the kernel. But the shape is the point: an eBPF program’s correctness is decided by what the verifier can prove, and the verifier’s vocabulary is types, readable ranges, and bounds. Every program in this book is written so that vocabulary is satisfied.

Hands-On Lab

# 1. Run the verifier-model demo: one rejection, one acceptance.
rustc code/ch02_ebpf_vm/verifier_notes.rs && ./verifier_notes

# 2. Meet the real verifier. Install bpftool, then try loading a
#    deliberately-bad program and read the log like a compiler error
#    (Chapter 14 decodes these lines).
sudo bpftool prog loadall /dev/null /sys/fs/bpf/demo 2>&1 | head

# 3. One-liner tracing: prove the hooks exist with bpftrace.
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_execve { @[comm] = count(); }'

The skill to practise: reading invalid access to packet, off=N size=M, R2 has K bytes readable and knowing the fix is a bounds check before the read - it is the single most common eBPF rejection you will ever see.

Summary

• An eBPF program is a register-machine program with no arbitrary memory access: stack, context, maps, and helpers are its entire world.
• Helpers are the whitelist of kernel functions programs may call; the verifier checks every call at load time.
• Maps are the only shared state - with other programs, the kernel, and userspace - and the map type must match the access pattern.
• The verifier proves type safety, bounds, termination, and reference correctness statically, then the JIT compiles the program to native code. Load is the moment of judgment; runtime is native speed.
• BTF + CO-RE let one compiled program relocate against the running kernel’s type information - no per-kernel recompiles.
• The lifecycle is compile, load, attach, run, unload - and the load step is where the verifier lives.

Next: Chapter 3 brings it to Rust - the Aya toolchain, the project layout, and the load/attach lifecycle in code you can actually run.

Chapter 3: Aya - eBPF Development in Rust

“One language, two worlds. The program that runs in the kernel and the program that feeds it maps - both Rust, both in one workspace, both reviewed by the same engineer.”

Chapter 2 described the machine. This chapter describes the workshop. Aya is the Rust eBPF library: the aya crate for the userspace side (loading ELF objects, creating maps, attaching programs, reading events) and the aya-bpf crate for the kernel side (the program context types, the map views, and the helper wrappers). Together with bpf-linker - a fork of LLVM’s ld.lld that understands eBPF relocations - they make it possible to write the entire eBPF stack in Rust. This chapter builds the project layout that every later chapter uses, and walks the load/attach lifecycle in real code: the workspace, the build chain, the kernel-side program, the userspace loader, and the error handling that turns verifier rejections into readable failures.

3.1 The Two-Crate Problem and the Aya Solution

An eBPF deployment has two halves that cannot be one crate:

• The kernel side compiles to the bpfel-unknown-none target - no standard library, no OS, no allocation, no std. It is a freestanding program that must fit the verifier’s rules. It lives in a crate of its own, built with cargo build --target bpfel-unknown-none, and linked with bpf-linker into an ELF object.
• The userspace side is an ordinary Rust program (with std, with tokio if you like, with clap for flags). It embeds the compiled kernel ELF as bytes (via include_bytes!), loads it with Ebpf::load, attaches the programs, and pumps the maps and ring buffers.

Aya’s answer to the two-world problem is a workspace with two crates plus a shared crate for the #[repr(C)] types that both sides read - the map keys, values, and event structs. One file defines the contract; both sides compile it. This is the same discipline as the shared header of a C kernel/userspace project, but with the type checker on both sides.

code/ch03_aya_toolchain/
├── Cargo.toml            # workspace: members = [kernel, userspace, common]
├── common/               # shared #[repr(C)] types: map keys, values, events
│   └── src/lib.rs
├── kernel/               # eBPF programs, target = bpfel-unknown-none
│   ├── Cargo.toml
│   ├── .cargo/config.toml    # sets the linker to bpf-linker
│   └── src/main.rs
└── userspace/            # the loader / control plane
    ├── Cargo.toml
    └── src/main.rs

3.2 The Build Chain: bpf-linker and the Toolchain

Compiling Rust to eBPF needs three pieces that are not default:

1. A target: bpfel-unknown-none (little-endian, freestanding). Install it with rustup target add bpfel-unknown-none.
2. A linker: bpf-linker (cargo install bpf-linker), a fork of lld that produces the ELF sections the kernel’s loader expects and that Aya understands. It is selected in the kernel crate’s .cargo/config.toml:

   [target.bpfel-unknown-none]
   linker = "bpf-linker"
   

3. A panic strategy: the kernel side cannot panic; panic = "abort" in [profile.release] and no_std in the crate root.

Source: code/ch03_aya_toolchain/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch03_aya_toolchain/kernel/src/main.rs
// Kernel side: a tracepoint program that records every execve into a
// HASH map, with the event struct shared from the `common` crate.
#![no_std]
#![no_main]

use aya_bpf::{
    macros::{map, tracepoint},
    maps::HashMap,
    programs::TracePointContext,
};
use common::ExecEvent;

// The tracepoint argument block layout for sched/sched_process_exec is
// defined by the kernel (include/trace/events/sched.h): pid at byte 0,
// comm at byte 16.
const PID_OFF: usize = 0;
const COMM_OFF: usize = 16;

#[map]
static EXECS: HashMap<u32, ExecEvent> = HashMap::with_max_entries(1024, 0);

#[tracepoint]
pub fn sched_process_exec(ctx: TracePointContext) -> i32 {
    let Ok(ev) = try_sched_process_exec(&ctx) else {
        return 0; // truncated tracepoint block: nothing to record
    };
    let mut ev = ev;
    let _ = EXECS.insert(&ev.pid, &mut ev, 0); // flags = 0: plain insert
    0
}

fn try_sched_process_exec(ctx: &TracePointContext) -> Result<ExecEvent, i32> {
    let pid: u32 = unsafe { ctx.read_at(PID_OFF)? };
    let comm: [u8; 16] = unsafe { ctx.read_at(COMM_OFF)? };
    Ok(ExecEvent { pid, comm })
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

Two details deserve comment. The #[map] macro declares the map and its ELF section; userspace will find it by name. And ctx.read_at is Aya’s checked read: it compiles to the bounds-checked load the verifier demands (Chapter 2), so the unsafe block is small and its invariant is exactly “the tracepoint block is at least this large” - a property of the kernel’s tracepoint ABI, not of our code.

3.3 The Userspace Loader: Ebpf::load and the Lifecycle

The userspace side is where the lifecycle of Chapter 2 becomes code:

Source: code/ch03_aya_toolchain/userspace/src/main.rs

// code/ch03_aya_toolchain/userspace/src/main.rs
// Userspace side: load the ELF, attach the tracepoint, read the map.
// Every step is a Result - no silent error handling (rule 4).
use aya::{maps::HashMap, programs::TracePoint, Ebpf};
use common::ExecEvent;
use std::error::Error;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const POLL_INTERVAL: std::time::Duration = std::time::Duration::from_secs(1);

fn main() -> Result<(), Box<dyn Error>> {
    // Load + attach. Ebpf::load runs the verifier (Ch.2) - a rejection
    // here is a load-time bug, not a runtime surprise.
    let mut ebpf = Ebpf::load(KERNEL_ELF)?;
    let program: &mut TracePoint = ebpf.program_mut("sched_process_exec")?.try_into()?;
    program.load()?;
    program.attach("sched", "sched_process_exec")?;
    println!("attached sched/sched_process_exec; Ctrl-C to exit");

    let mut execs: HashMap<_, u32, ExecEvent> = ebpf.map_mut("EXECS")?.try_into()?;

    loop {
        std::thread::sleep(POLL_INTERVAL);
        for key in execs.keys().collect::<Result<Vec<_>, _>>()? {
            let Some(ev) = execs.get(&key, 0)? else { continue };
            let comm = String::from_utf8_lossy(&ev.comm);
            println!("pid {} ({}) executed", ev.pid, comm.trim_end_matches('\0'));
        }
    }
}

Note the shape of the error handling: every fallible step is a Result, and the two most important failures - load (verifier) and attach (hook) - are mapped to messages that name the stage. The C++ book’s rule “no silent error handling” (CODING_STANDARDS rule 4) applies to eBPF doubly: a silent attach failure leaves you with a program that looks loaded and does nothing.

3.4 Attaching: Programs, Hooks, and Links

program.attach(...) is where the abstraction matters, because each program type has its own hook and its own attach signature:

Modern kernels (5.7+) attach through links (BPF_LINK_CREATE): a link is an object with a lifetime - when the link is dropped (process exit, or explicit detach), the program is detached automatically. Aya wraps this so that program.attach(...) keeps the program attached for as long as the returned link (or the process) lives. For a long-running daemon this is exactly what you want: no orphaned programs left attached after a crash.

3.5 The Shared Contract: One File, Two Compilers

The common crate deserves its own note, because it is where eBPF projects in Rust quietly succeed or loudly break. The kernel side and userspace side agree on map keys, values, and event structs only because both compile the same #[repr(C)] types. The rules:

• #[repr(C)] everywhere. Rust’s default struct layout is unspecified and may be reordered; the kernel and the userspace loader both need the stable C layout. Mark every type that crosses the boundary.
• Fixed-size integers. u32, u64, [u8; N], not usize (which is 64-bit on both sides here but a trap on other targets) and not String (which cannot cross at all - use [u8; N] and convert at the boundary).
• One source of truth. The struct lives in common; both crates depend on it. Copying the struct into both crates is how a kernel-side change silently desynchronises the userspace reader - the kind of bug that produces “map value has wrong size” at load time, or garbage events at runtime.

3.6 Running It: The Loop That Makes It Real

With the workspace above, the workflow is:

# kernel side: compile to eBPF ELF
cd code/ch03_aya_toolchain/kernel
cargo build --release --target bpfel-unknown-none

# userspace side: build and run (requires Linux, BTF enabled, root or
# CAP_BPF/CAP_PERFMON, and the tracepoint to exist on this kernel)
cd ../userspace
cargo run --release

When it works, you have done something that was impossible before Aya: a program running inside the Linux kernel, written entirely in Rust, attached and fed by a Rust control plane, in the same repository. When it fails, the failure will be one of the three this book teaches you to fix: a verifier rejection (Chapter 14), an attach error (hook name, privileges, or kernel config), or a contract mismatch between the two sides (Section 3.5).

Hands-On Lab

# 1. The toolchain (Linux):
rustup target add bpfel-unknown-none
cargo install bpf-linker

# 2. Build the kernel side, then the userspace side.
cd code/ch03_aya_toolchain/kernel
cargo build --release --target bpfel-unknown-none
cd ../userspace && cargo build --release

# 3. Run (root / CAP_BPF + CAP_PERFMON) and watch it load.
sudo ./target/release/userspace &
sudo bpftool prog list | grep sched_process_exec   # the tracepoint program

Then run ls a few times in another shell - every execve is recorded in the EXECS map, and the userspace loop prints the pid. That is the whole Chapter 3 lifecycle (load, attach, run, read maps) working end to end.

Summary

• eBPF in Rust is a two-crate problem: bpfel-unknown-none kernel programs plus an ordinary userspace loader, joined by a common crate of #[repr(C)] shared types.
• The build chain is rustup target add bpfel-unknown-none + bpf-linker; the kernel crate is no_std, panics abort, and the linker is set in .cargo/config.toml.
• The lifecycle in code is Ebpf::load (verifier) -> program.attach (hook) -> map/ring reads (data path) - every step a Result, every failure logged with the stage that failed.
• Attach signatures differ per program type; modern kernels attach through links, so detach is automatic when the process exits.
• The shared contract is the discipline: #[repr(C)], fixed-size types, one source of truth in common.

Next: Part II starts with the thing both sides of every eBPF project agree on - the maps. Chapter 4 is the complete map reference, from HASH to RINGBUF, with the access patterns that decide which one you need.

Chapter 4: BPF Maps - The Kernel/Userspace Contract

“A map is a promise: the kernel side promises a layout, the userspace side promises a reader, and both promise to agree on the key.”

Chapter 3 built the workspace; this chapter builds its shared vocabulary. BPF maps are the only state an eBPF program can hold across calls, the only channel between the kernel side and the userspace side, and the most common source of both performance wins and silent bugs. This chapter is the complete practical reference: the map types, their access patterns, the kernel-side and userspace-side APIs in Aya, and the decision procedure that picks the right map for a job. The capstone (Chapter 16) will use almost every map described here.

4.1 Why Maps Exist: State Without Trust

An eBPF program is stateless by default: each invocation starts fresh. But a DDoS filter must count packets, a load balancer must know its backends, a tracer must remember which syscalls it has seen. All of that state lives in maps: kernel-allocated tables, created at load time, typed by key and value, and accessible from both the program (through helpers) and userspace (through the bpf() syscall family that Aya wraps).

The security framing from Chapter 2 applies: the kernel owns the map’s memory, so a program can never corrupt adjacent memory through a map - the map lookup returns a pointer the verifier bounds-checks against the value size. What the kernel cannot do is fix your logic: a HASH map with the wrong key layout is perfectly safe and perfectly wrong.

4.2 The Map Family, Ranked by Access Pattern

The two starred rows are the ones to memorise, because they are the answer to the two questions every eBPF design asks: “where do I keep a hot counter?” (PERCPU_ARRAY) and “how do I stream events to userspace?” (RINGBUF).

4.3 HASH vs ARRAY: Lookup Cost and the Verifier

The choice between HASH and ARRAY is a cost model, not a taste. A hash lookup computes a hash, walks the bucket chain, and compares keys - on the order of tens of nanoseconds, with variance. An array lookup is one indexed load: a few nanoseconds, no variance, and it cannot fail the way a hash lookup can (no hash collision handling). The verifier knows the difference: an array lookup returns a pointer it can treat as always valid, while a hash lookup returns a pointer that may be NULL and must be checked.

In Aya the kernel-side APIs mirror this:

Source: code/ch04_bpf_maps/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch04_bpf_maps/kernel/src/main.rs
// Map choice in practice: ARRAY for fixed config, PERCPU_ARRAY for a hot
// counter, RINGBUF for events - the three default answers (Ch.4.2).
#![no_std]
#![no_main]

use aya_bpf::{
    macros::{map, xdp},
    maps::{Array, PerCpuArray, RingBuf},
    programs::XdpContext,
};

const MAX_IFACES: u32 = 8;
const XDP_DROP: u32 = 1;
const XDP_PASS: u32 = 2;

// Fixed configuration, one slot per interface. Array lookup cannot fail,
// so the verifier lets us use the pointer directly.
#[repr(C)]
pub struct IfaceConfig {
    pub drop: bool,        // drop packets on this interface?
    pub redirect_idx: u32, // XDP_TX via this queue (0 = none)
}
#[map]
static CFG: Array<IfaceConfig> = Array::with_max_entries(MAX_IFACES as u32, 0);

// Hot counter: per-CPU slots mean the kernel side never contends; the
// userspace side sums the CPUs on read.
#[map]
static PACKETS: PerCpuArray<u64> = PerCpuArray::with_max_entries(MAX_IFACES as u32, 0);

// Event stream: lock-free ring, kernel writes, userspace drains.
#[map]
static EVENTS: RingBuf = RingBuf::with_max_entries(4096, 0);

#[repr(C)]
pub struct PacketEvent {
    pub ifindex: u32,
    pub ts: u64, // bpf_ktime_get_ns; filled by the real capstone
}

#[xdp]
pub fn count_packets(ctx: XdpContext) -> u32 {
    let ifindex = ctx.if_index() % MAX_IFACES;

    // PERCPU_ARRAY: bump this CPU's slot - no cache-line contention.
    if let Some(slot) = PACKETS.get_ptr_mut(ifindex) {
        unsafe { *slot += 1 };
    }

    // ARRAY: config read without a possible NULL check.
    let cfg = unsafe { CFG.get_ptr(ifindex) };
    if cfg.is_some_and(|c| unsafe { (*c).drop }) {
        return XDP_DROP;
    }

    // RINGBUF: reserve - write - submit, or drop the event if full.
    let Ok(entry) = EVENTS.reserve::<PacketEvent>(0) else {
        return XDP_PASS; // ring full: the packet still passes
    };
    unsafe { entry.write(PacketEvent { ifindex, ts: 0 }) };
    entry.submit(0);

    XDP_PASS
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

Three patterns to internalise. Per-CPU for hot paths: the counter never touches a shared cache line on the kernel side (Chapter 13 measures the difference). Array for config: no NULL-check noise in the program, and the verifier is happy. Ring for events: reserve/write/submit is the whole API, and it is lock-free - the same ring discipline as the NIC of Chapter 1, with the kernel as producer and your userspace process as consumer.

4.4 The Userspace Side: Reads, Writes, and the Poll Loop

The userspace side of the contract is where Aya’s ergonomics show. Map handles come from the loaded Ebpf object by name; reads and writes are typed:

Source: code/ch04_bpf_maps/userspace/src/main.rs

// code/ch04_bpf_maps/userspace/src/main.rs
// Userspace side: configure the table, then read counters and events.
use aya::{
    maps::{Array, PerCpuArray, RingBuf},
    programs::Xdp,
    Ebpf,
};
use std::error::Error;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const IFACE: &str = "eth0";
const SLOT: u32 = 0;

#[repr(C)]
pub struct IfaceConfig {
    pub drop: bool,
    pub redirect_idx: u32,
}
#[repr(C)]
pub struct PacketEvent {
    pub ifindex: u32,
    pub ts: u64,
}

fn main() -> Result<(), Box<dyn Error>> {
    let mut ebpf = Ebpf::load(KERNEL_ELF)?;
    let program: &mut Xdp = ebpf.program_mut("count_packets")?.try_into()?;
    program.load()?;
    program.attach(IFACE, aya::programs::XdpFlags::default())?;

    // Config: slot 0 = do not drop, no redirect.
    let mut cfg: Array<_, IfaceConfig> = ebpf.map_mut("CFG")?.try_into()?;
    cfg.set(SLOT, IfaceConfig { drop: false, redirect_idx: 0 }, 0)?;

    // Per-CPU counters read per-CPU: sum across CPUs for the true total.
    let mut packets: PerCpuArray<_, u64> = ebpf.map_mut("PACKETS")?.try_into()?;
    let total: u64 = (0..num_cpus::get())
        .map(|cpu| packets.get(&SLOT, 0).ok().flatten().unwrap_or_default())
        .sum();
    println!("packets so far (sum over cpus): {total}");

    // Drain the event ring in batches (Ch.13 rule 4: batch the reads).
    let mut events: RingBuf = ebpf.map_mut("EVENTS")?.try_into()?;
    for item in events.iterator().take(10) {
        let ev: PacketEvent = item.read()?;
        println!("event: ifindex={} ts={}", ev.ifindex, ev.ts);
    }
    Ok(())
}

The detail worth staring at: per-CPU maps read per-CPU. get(&0, 0) returns only the current CPU’s value; summing across CPUs is a userspace responsibility. Aya offers PerCpuValues for reading all slots at once; either way, the moment you forget that a per-CPU map is a vector of values, one per CPU, your counters silently lose events. Chapter 13 returns to this with benchmark numbers.

4.5 Ring Buffers: The Modern Event Channel

BPF_MAP_TYPE_RINGBUF deserves its own section because it replaced the older perf_event_array for good reasons. The ring is lock-free (one producer side: the kernel program; one consumer side: your process), resizable-free (fixed at create), and it handles the memory-ordering concerns of the old perf buffers internally. The kernel side reserves a slot (reserve), fills it, and submits it; userspace polls the read side and advances the consumer index when done. If userspace is slow and the ring fills, the kernel side’s reserve fails - and you decide (drop the event, or drop the oldest) rather than the kernel deciding for you.

The ring is also where the NIC ring of Chapter 1 and the socket queue of Chapter 7 rhyme: the same head/tail producer-consumer shape, the same “keep up or lose data” rule. When the capstone ships events out of the kernel (Chapter 16), it ships them through this ring.

4.6 Map-in-Map and the Namespacing Pattern

ARRAY_OF_MAPS / HASH_OF_MAPS let a program select which map to use per packet - the classic namespacing pattern for multitenant systems: one outer map keyed by tenant id, whose values are map fds for that tenant’s counters. The kernel side does a two-step lookup (bpf_map_lookup_elem into the outer, then into the inner); the verifier treats the inner lookup like any other and the NULL check is mandatory. Cilium uses exactly this shape for per-endpoint and per-policy tables (Chapter 11). Use it when the number of tables is dynamic; prefer a single table with a composite key when the table count is fixed - one lookup beats two.

Hands-On Lab

# 1. Build and attach the XDP program that uses all three maps.
cd code/ch04_bpf_maps/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && cargo run --release -- eth0

# 2. Generate traffic, then inspect the maps' live state.
curl -s http://10.0.0.1/ >/dev/null &   # any traffic source
sudo bpftool map dump name PACKETS      # per-CPU slots: sum them yourself
sudo bpftool map show name PACKETS      # lookup/update counters per second
sudo bpftool map dump name EVENTS       # the ring events

The lesson to internalise: PACKETS is a per-CPU map, so bpftool map dump shows one value per CPU and the total is your job to sum - forget that and your counters silently undercount (Chapter 4.4, Chapter 13).

Summary

• Maps are the only shared state between kernel program and userspace: typed tables the kernel allocates and both sides access through helpers and syscalls.
• The default choices: PERCPU_ARRAY for hot counters, ARRAY for fixed config, HASH for per-flow state, LRU_HASH for bounded tables, RINGBUF for events.
• HASH lookups can fail (NULL check required); ARRAY lookups cannot. The verifier knows the difference and so should you.
• Per-CPU maps read per-CPU: userspace must sum across CPUs to get a true total.
• Ring buffers are lock-free, producer/consumer, drop-on-overflow-by- choice - the modern event channel, and the capstone’s observability backbone.
• Map-in-map (array/hash of maps) namespaces tables per tenant or endpoint; one composite key beats two lookups when the table count is fixed.

Next: Chapter 5 attaches the first program that runs per packet - XDP, at the very front of the path, before the skb even exists.

Chapter 5: XDP - eBPF at Line Rate

“XDP does not see packets. It sees bytes that used to be packets, before the kernel spent any time on them - and that is exactly its power.”

The XDP hook is where eBPF earns its reputation. Attached at the very front of the receive path - in the driver’s poll context, before an skb is allocated, before GRO, before netfilter, before anything that costs money - an XDP program sees the raw DMA’d frame and must decide its fate in the per-packet budget of Chapter 1. Drop, pass, transmit back out the same interface, or redirect to another interface, another CPU, or an AF_XDP socket. This chapter is the practical XDP reference: the program, the actions, the packet-parsing discipline, and the Aya attach path.

5.1 Where XDP Runs and What It Sees

Recall the path of Chapter 1: the NIC DMAs a frame into the ring; the driver’s NAPI poll loop wakes; normally the driver builds an skb and hands it to the stack. XDP runs in that poll loop, on the raw frame, before the skb exists. The program receives a context with two pointers - data and data_end - delimiting the frame in DMA memory, plus the receive queue’s index. Everything the program needs (MAC addresses, IP headers, ports) it must parse itself from those bytes, with the bounds discipline of Chapter 2: every read checked against data_end.

Three properties follow:

1. No allocation. The frame is in memory the NIC filled; the program neither allocates nor frees. This is why an XDP drop can cost ~100 ns while the same drop in the stack costs thousands.
2. No stack, no protocol state. There is no skb, no routing decision, no conntrack entry, no socket. XDP is pre-protocol. If your decision needs connection state, either build it in maps yourself or use a later hook.
3. The driver must support it. XDP attaches to a driver’s native path (native mode). Generic XDP exists as a fallback that runs in the stack, and offloaded XDP runs in the NIC’s firmware; for performance you want native, and ethtool -L tells you whether your NIC and driver support it.

5.2 The Actions: A Decision, Not a Filter

An XDP program returns one of five actions, and the action is the whole output of the program:

The framing that matters: XDP is a decision point, not a packet filter. DROP is the DDoS use case (drop bad traffic at the NIC, before it costs anything). REDIRECT is the load-balancing use case (steer a packet to a backend interface without the stack touching it) and the AF_XDP use case (hand the frame to a userspace poller, zero-copy). PASS is the honest fallback: “I looked, I do not want to deal with this packet - let the stack have it.”

Source: code/ch05_xdp/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch05_xdp/kernel/src/main.rs
// The canonical XDP program shape: parse once, look up once, act.
// Fail-open everywhere - an unparseable or unknown packet passes (Ch.5.2).
#![no_std]
#![no_main]

mod parse;

use aya_bpf::{
    macros::{map, xdp},
    maps::HashMap,
    programs::XdpContext,
};
use aya_bpf::bindings::xdp_action::*;

const MAX_BACKENDS: u32 = 64;

// Backend table: key = dst port, value = where to send it.
#[repr(C)]
pub struct Backend {
    pub ifindex: u32, // redirect target interface
    pub cpu: u32,     // redirect target CPU (reserved)
}
#[map]
static BACKENDS: HashMap<u16, Backend> = HashMap::with_max_entries(MAX_BACKENDS, 0);

#[xdp]
pub fn backend_lb(ctx: XdpContext) -> u32 {
    // Turn (data, data_end) into a slice once; all parsing is slice reads.
    let frame = unsafe { parse::frame(ctx.data(), ctx.data_end()) };

    // Parse once (eth + ip + tcp). None means "not ours": pass it on.
    let Some(t) = parse::tcp_tuple(frame) else {
        return XDP_PASS;
    };

    // One map lookup by dst port. Hash lookups can fail (Ch.4): no entry
    // means "not a service we know" - pass it on.
    match BACKENDS.get(&t.dport) {
        Some(be) => aya_bpf::helpers::bpf_redirect(be.ifindex, 0),
        None => XDP_PASS,
    }
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

Read the program as a conversation with the verifier. Every parse step begins with a bounds check; every lookup handles the NULL case; every branch that cannot decide returns XDP_PASS rather than dropping a packet that might be legitimate. This fail-open discipline - pass when unsure - is what separates a network program from a network accident.

5.3 The Bounds Discipline: data, data_end, and the Verifier

The single most common verifier rejection in XDP programming is the one shown in Chapter 2: reading past data_end. The rule is absolute: every packet access must be provably within [data, data_end]. In practice that means the program structure is fixed:

check: start + offset + size <= data_end      -> then read

Aya’s XdpContext exposes data() and data_end(), and the pattern above

• compute the needed extent, compare, then dereference - is the pattern the verifier accepts. Two subtleties that bite:

• Helper calls can invalidate packet pointers. After a call that may reallocate or grow the frame (e.g. bpf_xdp_adjust_head), the old data pointer is no longer usable and must be recomputed. Aya’s types enforce this: the context’s data pointers are only valid until such a call, and the compiler (like the verifier) complains if you hold them across one.

• Do not parse twice what you can parse once. Each bounds check costs a comparison; a frame with three headers wants three checks, not nine. Parse the headers in order, carrying the offset forward - the pattern the capstone uses.

5.4 AF_XDP: The Zero-Copy Path to Userspace

XDP’s most powerful redirect target is AF_XDP: a socket type that gives userspace zero-copy access to the frames XDP redirects to it. The architecture is the rings of Chapter 1 done twice: the NIC’s RX ring, and a set of UMEM rings (fill, rx, tx, completion) shared between kernel and userspace. An XDP program does XDP_REDIRECT into an AF_XDP socket’s rx ring; userspace reads frames directly from the UMEM without a copy, and recycles buffers through the fill ring.

AF_XDP is the escape hatch that keeps XDP honest: when your packet processing outgrows what the verifier’s instruction budget allows, or when you need userspace libraries for parsing (TLS, HTTP/2, your own L7), you do not leave the kernel - you redirect into userspace and keep the zero-copy path. This is exactly the architecture of modern high-performance proxies and of Cilium’s service mesh when L7 processing must happen (Chapter 12).

5.5 Attaching XDP from Aya

Source: code/ch05_xdp/userspace/src/main.rs

// code/ch05_xdp/userspace/src/main.rs
// Userspace side: load, attach, fill the backend table.
use aya::{maps::HashMap, programs::Xdp, Ebpf};
use std::error::Error;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const DEFAULT_IFACE: &str = "eth0";
const SERVICE_PORT: u16 = 8080;
const BACKEND_IFINDEX: u32 = 2;

#[repr(C)]
pub struct Backend {
    pub ifindex: u32,
    pub cpu: u32,
}

fn main() -> Result<(), Box<dyn Error>> {
    let iface = std::env::args().nth(1).unwrap_or_else(|| DEFAULT_IFACE.to_string());

    let mut ebpf = Ebpf::load(KERNEL_ELF)?;
    let program: &mut Xdp = ebpf.program_mut("backend_lb")?.try_into()?;
    program.load()?;
    program.attach(&iface, aya::programs::XdpFlags::default())?;
    println!("XDP attached to {iface}");

    let mut backends: HashMap<_, u16, Backend> = ebpf.map_mut("BACKENDS")?.try_into()?;
    backends.insert(
        SERVICE_PORT,
        Backend { ifindex: BACKEND_IFINDEX, cpu: 0 },
        0,
    )?;
    println!("backend table: port {SERVICE_PORT} -> ifindex {BACKEND_IFINDEX}");
    Ok(())
}

Note what is not in this program: no packet code. The control plane fills maps; the data plane makes decisions. Keeping the two planes separate is the Chapter 13 methodology in miniature - and it is why the capstone can reconfigure a running data path by updating a map, not by reloading a program.

5.6 XDP in Production: DDoS, LB, and the Security Story

XDP’s production track record is the strongest argument for learning it: Facebook’s Katran (an XDP load balancer) and Cloudflare’s DDoS mitigation both drop or redirect tens of millions of packets per second on commodity hardware - numbers the stack cannot touch because every packet the stack sees has already paid for an skb. Katran in particular is the production ancestor of the capstone’s XDP load balancer: hash the flow tuple, look up a backend in a map, XDP_REDIRECT to the backend’s interface. The DDoS story is even simpler: a HASH map of bad addresses, XDP_DROP on match, fail-open otherwise. Both are the same program shape you just wrote - parse, look up, act - at a scale that only the front-of-the-path hook can reach.

Hands-On Lab

# 1. Build and attach the XDP load balancer to a veth pair.
cd code/ch05_xdp/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && sudo cargo run --release -- veth0

# 2. Prove it runs before the stack: the JIT output is a few dozen
#    instructions, and the drop never reaches tcpdump.
sudo bpftool prog dump jited name backend_lb
sudo tcpdump -i veth0 -nn -c 5          # packets to the service are gone

# 3. Measure the budget (Chapter 13): pktgen at 1 Mpps, then read the
#    per-program counters.
sudo bpftool prog show name backend_lb  # runtime + the 'drop' counter

The verification loop is: traffic in, map counter up, tcpdump silent, JIT small. All four together are what “XDP at line rate” means.

Summary

• XDP runs in the driver’s poll loop, on the raw DMA’d frame, before skb allocation - no allocation, no stack, no protocol state.
• The output is one of five actions: ABORTED, DROP, PASS, TX, REDIRECT. It is a decision point, not a filter.
• Bounds discipline is the whole game: every read checked against data_end, helper calls invalidate packet pointers, parse-once.
• REDIRECT to AF_XDP gives userspace zero-copy access - the escape hatch for L7 processing beyond the verifier’s budget.
• Attach from Aya is XdpFlags::default() on an interface; control plane (maps) and data plane (program) stay separate.
• Production XDP (Katran, Cloudflare) is the same shape you wrote, at tens of Mpps - the proof that the front-of-path hook is worth it.

Next: Chapter 6 crosses to the other side of the path - the traffic control hooks that run after the stack has allocated the skb.

Chapter 6: Traffic Control & the Socket Filter Hooks

“XDP sees the packet before the kernel believes in it. tc sees the packet after the kernel has invested in it - and that investment buys you the whole protocol stack on both sides of the filter.”

Chapter 5 took the hook at the front door. This chapter takes the hooks between the stack’s halves: the traffic control (tc) layer, which runs egress after the stack has produced an skb and ingress before the stack consumes it, plus the classic socket filter hook where the Berkeley Packet Filter was born. Where XDP is a pre-protocol decision point, tc is a post-protocol inspection point: it sees skbs with routing, conntrack and socket context - and it can modify, redirect, drop, or rate them. This is the hook Cilium uses for most of its data path, and the hook where the trade-offs between “as early as possible” and “as informed as possible” become concrete.

6.1 The Two Worlds of Traffic Control

The tc subsystem predates eBPF by decades: it is the kernel’s queueing and shaping machinery (tc qdisc, tc filter). Its eBPF integration gave the ancient clsact class a modern superpower: an eBPF program attached as a classifier can decide a packet’s fate and return an action the kernel obeys. The modern attach point is the clsact qdisc, which provides two hook points per interface:

• ingress: runs on receive, after GRO and the early driver path, on the way to protocol dispatch (Chapter 1’s netif_receive_skb).
• egress: runs on transmit, after the stack has produced the skb (routing, socket, TCP segmentation decisions all made), before the driver’s transmit path.

Each hook runs an ordered list of filters; a clsact eBPF classifier returns TC_ACT_OK (accept), TC_ACT_SHOT (drop), TC_ACT_REDIRECT (redirect), or a few rarer actions, and may edit the skb in place.

6.2 XDP vs tc: The Decision Table

Every networking eBPF design eventually asks: XDP or tc? The answer is a table, not a preference:

The rule of thumb that the rest of the book uses: drop as early as you can justify; decide as late as you must. If the decision is “is this address on the ban list?” - XDP. If the decision is “does this connection belong to this pod’s policy?” - tc, because the socket and conntrack state that makes the answer correct only exists there.

6.3 The tc Program in Aya

Source: code/ch06_tc_hooks/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch06_tc_hooks/kernel/src/main.rs
// A tc ingress classifier: one composite-key lookup against an allowlist.
#![no_std]
#![no_main]

mod parse;

use aya_bpf::{
    macros::{classifier, map},
    maps::HashMap,
    programs::SchedClassifierContext,
};
use aya_bpf::bindings::tc_actions::*;

const MAX_RULES: u32 = 1024;
const ACTION_ALLOW: u32 = 1;

#[repr(C)]
pub struct Rule {
    pub action: u32, // 1 = allow, 0 = drop
}
#[map]
static ALLOW: HashMap<u64, Rule> = HashMap::with_max_entries(MAX_RULES, 0);

#[classifier]
pub fn ingress_check(ctx: SchedClassifierContext) -> i32 {
    // tc contexts hand the frame as an Option; a missing skb is not ours
    // to judge - the stack sorts it out.
    let Some(start) = ctx.data() else {
        return TC_ACT_OK;
    };
    let frame = unsafe { parse::frame(start, ctx.data_end()) };

    // Parse the composite key; None means non-TCP or truncated - pass.
    let Some(key) = parse::allow_key(frame) else {
        return TC_ACT_OK;
    };

    // Fail-open: a missing entry allows. An allowlist that was never
    // populated must not black-hole the interface (Ch.6.3).
    match ALLOW.get(&key) {
        Some(rule) if rule.action == ACTION_ALLOW => TC_ACT_OK,
        Some(_) => TC_ACT_SHOT,
        None => TC_ACT_OK,
    }
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

The program is the XDP shape of Chapter 5 with one crucial difference: it runs on an skb, so everything the stack already did - GRO merging, validation, routing - is behind it, and the socket-level helpers of later chapters (redirect into a sockmap, edit the skb) are available to it. Note also the fail-open choice in the last match arm: an allowlist whose table is empty would otherwise block the entire interface. That decision - what happens when the map is empty, stale, or half-written - is a control-plane contract, and Chapter 14 tests exactly this case.

6.4 Attaching tc from Aya: The clsact Dance

tc attachment requires creating the clsact qdisc first; Aya wraps this but the sequence is worth seeing once, because it is the most common attach failure in the book:

Source: code/ch06_tc_hooks/userspace/src/main.rs

// code/ch06_tc_hooks/userspace/src/main.rs
// Userspace side: create clsact, attach to ingress, fill the allowlist.
use aya::programs::{tc::SchedClassifierAttachMode, SchedClassifier};
use aya::Ebpf;
use std::error::Error;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const DEFAULT_IFACE: &str = "eth0";
const ALLOWED_IP: u32 = 0x0a00_0001; // 10.0.0.1
const ALLOWED_PORT: u16 = 443;
const ACTION_ALLOW: u32 = 1;

#[repr(C)]
pub struct Rule {
    pub action: u32,
}

fn main() -> Result<(), Box<dyn Error>> {
    let iface = std::env::args().nth(1).unwrap_or_else(|| DEFAULT_IFACE.to_string());

    let mut ebpf = Ebpf::load(KERNEL_ELF)?;
    let program: &mut SchedClassifier = ebpf.program_mut("ingress_check")?.try_into()?;
    program.load()?;
    // clsact must exist before any classifier attaches (Ch.6.4); Aya
    // creates it as part of the attach.
    program.attach(&iface, SchedClassifierAttachMode::Ingress)?;
    println!("tc ingress classifier attached to {iface}");

    let key = (u64::from(ALLOWED_IP) << 16) | u64::from(ALLOWED_PORT);
    let mut allow: aya::maps::HashMap<_, u64, Rule> = ebpf.map_mut("ALLOW")?.try_into()?;
    allow.insert(key, Rule { action: ALLOW }, 0)?;
    println!("allowlist: 10.0.0.1:443 allowed");
    Ok(())
}

The clsact requirement is why tc eBPF demos always show a tc qdisc add dev eth0 clsact step: the qdisc is the hook’s mount point. If you forget it, attach fails with a clear error; if you attach to a veth or a bond, check that the underlying driver supports clsact on that device type.

6.5 Editing the skb: bpf_skb_store_bytes and the Verifier’s Rules

Unlike XDP, a tc program can modify the packet: bpf_skb_store_bytes writes into the skb with the kernel managing the buffer, and bpf_skb_change_proto / bpf_skb_adjust_room grow or shrink it. This is the machinery of NAT, encapsulation (VXLAN/Geneve), and policy rewriting. The verifier’s rules are the price: after a change helper, the old packet pointers are invalid (Chapter 5’s rule, applied to skb), and the checksum helpers (bpf_l4_csum_replace) must be called in the right order. The discipline is identical to XDP - recompute bounds after every mutation - and the consequence of getting it wrong is a packet the kernel forwards with a stale checksum, which is worse than dropping it.

Cilium’s use of these helpers is the reference implementation to study: the bpf_lxc program that implements pod policy edits and redirects is a few hundred lines of exactly this shape - parse, lookup policy in maps, edit if needed, return the action. Chapter 11 walks it.

6.6 The Socket Filter: Where BPF Was Born

The oldest BPF hook is the socket filter: a program attached to a socket that runs on every packet queued to that socket, and whose return value decides how many bytes of the packet the socket sees (return 0 to drop it, return the header length to peek, return the full length to accept). This is the direct descendant of the original 1992 Berkeley Packet Filter of tcpdump: the filter language that grew into eBPF. In Aya it is the SockFilter program type, attached per socket.

Its role today is modest but real: cheap per-socket filtering (drop traffic before userspace sees it), traffic accounting, and the cgroup variant (Chapter 9) that filters every socket in a cgroup at once. The important lesson is historical: the socket filter is where the idea of running a small, verified program inside the kernel’s data path was proven safe enough to generalise into everything else in this book. When someone asks “why does the kernel trust eBPF?”, the honest answer begins “because it has been running packet filters since 1992.”

Hands-On Lab

# 1. Build and attach the tc ingress classifier (Aya creates clsact).
cd code/ch06_tc_hooks/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && sudo cargo run --release -- veth0

# 2. Confirm the hook is a sched_cls program on the veth.
sudo bpftool prog list | grep sched_cls
sudo tc filter show dev veth0 ingress    # the classic view of the same hook

# 3. Remove the qdisc and watch the attach contract break.
sudo tc qdisc del dev veth0 clsact       # next attach fails: mount point gone

The discipline: an skb with a populated allowlist gets one hash lookup; an empty allowlist still passes everything (fail-open). Both are visible in bpftool map dump name ALLOW - the map is the program’s visible state.

Summary

• tc is the post-stack hook: ingress before protocol dispatch, egress after the stack has produced the skb, attached via the clsact qdisc.
• The XDP-vs-tc table: drop early, decide late. XDP is pre-protocol and cheap; tc sees routing, sockets and conntrack and can edit packets.
• The tc program is the XDP shape plus skb editing: bpf_skb_store_bytes and friends, with pointer invalidation and checksum rules as the price.
• The clsact qdisc must exist before attach - the classic tc attach failure, wrapped by Aya but worth understanding.
• The socket filter is the ancestor of it all: per-socket, decide how many bytes the socket sees, since 1992.

Next: Part III leaves the packet alone and goes deep into the protocol layer - Chapter 7 is the full TCP/IP receive path, from the ACK in the NIC to the recvfrom in your hand.

Chapter 7: TCP/IP, Sockets & the Receive Path

“TCP is not a protocol for moving bytes. It is a protocol for agreeing, byte by byte, on what has been received - and the receive path is where that agreement is enforced, at line rate, in softirq context.”

Chapters 5-6 put eBPF on the packet path. This chapter puts the protocol under a microscope: what the kernel actually does with a TCP segment from the moment GRO hands it over until recvfrom returns your bytes. This is the layer where sockets, conntrack, timers, retransmits, and backpressure live - and it is the layer eBPF observes through kprobes and tracepoints and modifies through sockmap (Chapter 8) and cgroup hooks (Chapter 9). A systems engineer who cannot trace a segment through this path is debugging blind.

7.1 The Socket: The Kernel’s File of the Network

A socket is a pair of queues - receive and send - wrapped in state, owned by a file descriptor, and created by the socket() syscall. The receive path of Chapter 1 ends here: the stack queues the incoming data into the socket’s receive buffer (sk_rcvbuf), and your recvfrom copies it out. The crucial fact for everything that follows: the socket is where the kernel’s protocol processing meets the process’s syscalls, and every eBPF program that touches sockets - sockmap, socket filters, cgroup hooks - is attached at this meeting point.

A TCP socket’s state machine matters less for this book than its queues: sk_receive_queue (data waiting for recv), sk_write_queue (data waiting for the network), and the out-of-order queue. The receive queue is an sk_buff_head, and the queue discipline is the lock discipline: the socket lock protects it, and the softirq that delivers the segment takes that lock. Contention on the socket lock - one flow per socket, one CPU per flow via RSS - is why TCP throughput is fundamentally single-queue per socket, and why sockmap’s redirection (Chapter 8) is such a big deal: it moves the socket-level decision into the kernel without touching the queue.

7.2 The Receive Path, From GRO to recvfrom

Follow one segment through the path of Chapter 1, now with protocol detail:

1. GRO (Chapter 1) merges back-to-back segments of one flow into a larger skb. The TCP layer usually sees runs, not segments.
2. Protocol dispatch routes the skb by ethertype to the IP layer; netfilter hooks fire (PREROUTING, conntrack, INPUT). Conntrack creates or updates the connection entry - the table Cilium replaces with BPF maps in Chapter 11.
3. TCP processing (tcp_v4_rcv) looks up the socket by the 4-tuple (src ip, src port, dst ip, dst port) in the per-netns listen/established hash tables. The segment is validated: sequence numbers, checksum, window. ACKs are handled immediately - the receive window slides, timers are updated - and then the data is queued.
4. The socket queue: the data lands in sk_receive_queue, and if a process is blocked in recvfrom, it is woken. The kernel copies the data out under the socket lock and releases the skb.
5. Your syscall: recvfrom returns; the bytes are in your buffer. The default path cost - from NIC to your buffer - is the 2-10 us of Chapter 1.

The two things eBPF observes here are function entry points (kprobes) and tracepoints - kernel-instrumented points with stable, documented argument blocks. The most useful for network debugging: tcp:tcp_rcv_space_adjust (receive buffer behaviour), tcp:tcp_retransmit_skb (the retransmit event, i.e. the network is losing), sock:inet_sock_set_state (every socket state transition - the backbone of Cilium’s Hubble flow logging), and skb:kfree_skb (every dropped packet, with the drop reason).

7.3 A Tracing Program: Watching Retransmits with Aya

Source: code/ch07_tcp_receive_path/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch07_tcp_receive_path/kernel/src/main.rs
// Observe the receive path: count TCP retransmits per destination, and
// ship one event per retransmit through the ring buffer.
#![no_std]
#![no_main]

use aya_bpf::{
    macros::{map, tracepoint},
    maps::{HashMap, RingBuf},
    programs::TracePointContext,
};

const MAX_DESTS: u32 = 4096;
// struct sock: skc_daddr sits at offset 32 in the common fields on the
// kernels this book targets; CO-RE resolves it in production code (Ch.2).
const SK_DADDR_OFF: usize = 32;

// Per-destination retransmit counter, keyed by dst ip (host byte order).
#[map]
static RETRANS: HashMap<u32, u32> = HashMap::with_max_entries(MAX_DESTS, 0);

#[repr(C)]
pub struct RetransEvent {
    pub dst_ip: u32,
    pub ts: u64,
}
#[map]
static EVENTS: RingBuf = RingBuf::with_max_entries(4096, 0);

// tcp:tcp_retransmit_skb fires on every retransmission, in softirq
// context, with a stable argument block: (sk, skb).
#[tracepoint]
pub fn tcp_retransmit_skb(ctx: TracePointContext) -> i32 {
    let Ok(dst_ip) = try_retransmit(&ctx) else {
        return 0; // truncated tracepoint block: nothing to record
    };

    // Count, then emit one event. Both map ops are best-effort.
    let mut n = RETRANS.get(&dst_ip).unwrap_or_default() + 1;
    let _ = RETRANS.insert(&dst_ip, &mut n, 0);

    let Ok(entry) = EVENTS.reserve::<RetransEvent>(0) else {
        return 0; // ring full: drop the event, keep counting
    };
    unsafe {
        entry.write(RetransEvent {
            dst_ip,
            ts: aya_bpf::helpers::bpf_ktime_get_ns(),
        });
    }
    entry.submit(0);
    0
}

fn try_retransmit(ctx: &TracePointContext) -> Result<u32, i32> {
    let sk: u64 = unsafe { ctx.read_at(0)? };
    // Reach the socket's destination field as an unaligned u32 read; the
    // offset is a kernel-version detail CO-RE resolves in production.
    let p = (sk as *const u8).add(SK_DADDR_OFF) as *const u32;
    Ok(u32::from_be(unsafe { p.read_unaligned() }))
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

This is the observability face of eBPF: the program does not change the path, it reports on it. The retransmit event is the signal that the network is losing packets - the single most important metric for any TCP-dependent system, and the reason Cilium’s Hubble and every serious network observability stack are built on exactly these tracepoints.

7.4 GRO, GSO and the Segment Illusion

Chapter 1 introduced GRO/GSO as a cost optimisation; here is the protocol consequence. Because GRO merges segments before TCP sees them, the TCP layer’s receive window accounting and ACK generation work on merged runs; because GSO re-segments on transmit, the wire sees standard MTU frames again. The practical rule for eBPF writers: an skb at tc ingress may represent several wire segments, so per-segment assumptions (one ACK per skb, one header per skb) are wrong. Read the lengths and count segments from the headers, never assume.

7.5 Backpressure: What Happens When You Are Slow

The receive path has a built-in negotiation between the stack and your process, and it is the last piece of the model. If your process does not read fast enough, the socket’s receive buffer fills; the stack stops accepting data for that socket (the window closes, the sender throttles) and eventually drops. This is not a failure; it is the protocol working. TCP is a flow-control protocol; the receive buffer is the flow-control mechanism. When you see recvfrom latency climbing, the question is never “why is the kernel slow” but “where is the queue filling” - and the answer is either your process’s read loop, the socket buffer size, or the NIC queue of Chapter 1. eBPF sees all three: the socket queues via sockmap (Chapter 8), the cgroup’s socket usage via cgroup hooks (Chapter 9), and the NIC queues via XDP/tc counters.

Hands-On Lab

# 1. Load the retransmit counter on the host.
cd code/ch07_tcp_receive_path/kernel && cargo build --release --target bpfel-unknown-none

# 2. Induce loss deliberately, then watch the events arrive.
sudo tc qdisc add dev eth0 root netem loss 10%   # drop 10% of packets
# (run a download through eth0 - the retransmits fire)
sudo bpftool map dump name RETRANS               # per-destination counts

# 3. Cross-check with bpftrace - the same tracepoint, one line.
sudo bpftrace -e 'k:tcp_retransmit_skb { @ = count(); }'
sudo tc qdisc del dev eth0 root netem            # clean up

The moment to notice: the tracepoint fires in softirq context, on the RSS-assigned CPU, for a connection that may belong to another process. That is why observability programs are attached from outside the process (Chapters 7, 15).

Summary

• A socket is two queues plus state, owned by an fd - the meeting point of kernel protocol processing and process syscalls.
• The receive path is GRO -> IP/netfilter/conntrack -> TCP lookup and queue -> socket lock -> syscall copy, and eBPF observes each stage through tracepoints (tcp_retransmit_skb, inet_sock_set_state, kfree_skb).
• kprobes/tracepoints are the observability hooks: they report, they do not modify - and they are what Hubble and every serious network observability stack are built on.
• GRO merges segments; GSO re-segments. An skb is not a segment; read lengths from headers, never assume.
• Backpressure is the protocol working: the receive buffer is the flow-control mechanism, and slow readers throttle the sender by design.

Next: Chapter 8 changes the path instead of reporting on it - sockmap and SK_MSG, eBPF’s in-kernel socket redirect, the mechanism behind Cilium’s service mesh.

Chapter 8: Sockmap & SK_MSG - eBPF in the Socket Layer

“A proxy is a program that moves bytes between sockets. sockmap is a program that moves bytes between sockets without leaving the kernel - no copy to userspace, no syscall, no proxy process at all.”

Chapter 7 ended at the socket. This chapter enters it. sockmap is a BPF map type whose values are sockets, and SK_MSG / SK_SKB are the program types attached to it: they run in the socket layer, on data being sent to or received from a socket in the map, and they can redirect that data to another socket in the map - in the kernel, without a round trip through userspace. This is the mechanism behind Cilium’s socket-level load balancing and its service mesh data path: when a pod talks to a service, the kernel itself rewrites the destination socket, and no proxy process ever touches the bytes. This chapter is the complete practical treatment: the map, the program types, the redirect helpers, and the semantics that make it correct.

8.1 The Problem: Why Proxies Are Slow

A classic service proxy (Envoy, HAProxy, an L4 load balancer) does the following per connection: accept on the front socket, read into a userspace buffer, write to the back socket. Every byte crosses the kernel/userspace boundary twice per direction - two read/write pairs, four context switches in the worst case, two copies - and the proxy process consumes a core per some thousands of connections. The kernel, by contrast, already has both sockets. The question sockmap answers is: why should the bytes visit userspace at all when the kernel can move them directly?

8.2 The sockmap: A Map Whose Values Are Sockets

A sockmap (or its hash twin, sockhash) is a BPF map keyed by anything you like - typically the service tuple or a connection id - whose values are sockets. The socket is inserted from userspace by passing the file descriptor; the kernel stores the real socket object. Two things make this special:

1. Programs attach to the map, not to an interface. A SK_MSG program is attached to a sockmap with a direction (BPF_SK_MSG_VERDICT), and it runs on every send on a socket in that map. A SK_SKB program runs on received data instead.
2. Redirect is map-mediated. The program calls bpf_msg_redirect_map (SK_MSG) or bpf_sk_redirect_map (SK_SKB) with a key; the kernel looks up the target socket in the same map and delivers the data to it - queuing it on the target’s receive queue, with the flow-control, ordering, and wake-up semantics of a normal receive.

The mental model: the sockmap is a routing table for sockets. The program is the routing logic. The data plane of a sockmap proxy is entirely inside the kernel.

8.3 SK_MSG: The Send-Side Program

The SK_MSG program runs in the send path of a socket in the map, before the data is handed to the protocol layer. Its context is the message being sent; its decision is a verdict: SK_MSG_PASS (let the send proceed normally) or SK_MSG_REDIRECT (send these bytes to another socket in the map, via bpf_msg_redirect_map).

Source: code/ch08_sockmap/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch08_sockmap/kernel/src/main.rs
// SK_MSG redirect: every send on a socket in the map is redirected to a
// peer socket looked up by a per-connection key in the same map.
#![no_std]
#![no_main]

use aya_bpf::{
    macros::{map, sk_msg},
    maps::SockHash,
    programs::SkMsgContext,
};
use aya_bpf::bindings::bpf_msg_action::*;

const MAX_CONNS: u32 = 65536;
const CB_CONN_ID: usize = 0; // control plane stamps this at setup (Ch.8.3)

// Key = connection id, value = the peer socket for that direction.
#[map]
static CONNS: SockHash<u32> = SockHash::with_max_entries(MAX_CONNS, 0);

#[sk_msg]
pub fn msg_redirect(ctx: SkMsgContext) -> i32 {
    let conn_id = unsafe { ctx.cb(CB_CONN_ID) as u32 };

    // One map-mediated handoff, in the kernel. A lookup miss must never
    // eat data: any error verdict degrades to PASS (fail-open).
    let rc = unsafe {
        aya_bpf::helpers::bpf_msg_redirect_hash(
            &mut ctx as *mut SkMsgContext as *mut _,
            &mut CONNS as *mut SockHash<u32> as *mut _,
            &conn_id as *const u32 as *const _,
            0,
        )
    };
    if rc == SK_MSG_PASS || rc == SK_MSG_REDIRECT {
        rc as i32
    } else {
        SK_MSG_PASS
    }
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

Two details carry the whole design. The connection id in the cb: the skb control block (32 bytes of per-packet scratch, Chapter 6) is how the kernel side remembers which connection this send belongs to without a per-send map lookup on the tuple - the control plane writes it at connection setup, the program reads it per send. The fail-open fallback: bpf_msg_redirect_hash returns SK_MSG_PASS (accept) or SK_MSG_REDIRECT (done); anything else is an error, and the program passes the data through rather than dropping it. A sockmap misroute must never become a silent data loss.

8.4 The Userspace Side: Building the Socket Table

The userspace side of sockmap is where the sockets come from - and where the correctness lives. The control plane must own both sockets: the front (client-facing) and the back (upstream) socket of every proxied connection. In the classic pattern, a userspace program accepts the incoming connection and dials the backend - then inserts both sockets into the sockhash and hands the data path to the kernel:

Source: code/ch08_sockmap/userspace/src/main.rs

// code/ch08_sockmap/userspace/src/main.rs
// The control plane: accept a connection, dial the backend, insert both
// sockets into the sockhash, and let the kernel move the bytes (Ch.8.4).
use aya::{maps::SockHash, programs::SockMap, Ebpf};
use std::error::Error;
use std::net::TcpListener;
use std::os::fd::AsRawFd;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const FRONT_ADDR: &str = "127.0.0.1:8080";
const BACK_ADDR: &str = "127.0.0.1:9090";

fn main() -> Result<(), Box<dyn Error>> {
    let mut ebpf = Ebpf::load(KERNEL_ELF)?;

    // Attach the SK_MSG program to the sockhash: it runs on every send
    // on a socket inserted into this map.
    let prog: &mut SockMap = ebpf.program_mut("msg_redirect")?.try_into()?;
    prog.load()?;
    prog.attach(&aya::programs::SockMapAttachType::MsgVerdict)?;

    let mut conns: SockHash<_, u32> = ebpf.map_mut("CONNS")?.try_into()?;
    let front = TcpListener::bind(FRONT_ADDR)?;
    let back = TcpListener::bind(BACK_ADDR)?;
    println!("sockmap control plane ready on {FRONT_ADDR}");

    for (i, (front_sock, _)) in front.incoming().enumerate() {
        let (back_sock, _) = back.accept()?;
        let conn_id = i as u32;

        // Both sockets under the same key; redirects stay in the kernel.
        conns.insert(conn_id, front_sock.as_raw_fd(), 0)?;
        conns.insert(conn_id ^ 1, back_sock.as_raw_fd(), 0)?;
        println!("connection {conn_id}: kernel now moves the bytes");
    }
    Ok(())
}

The pattern is a hybrid: userspace for the connection establishment (where the complexity lives: TLS, auth, backend discovery), kernel for the steady-state data movement (where the speed matters). This is precisely the architecture Cilium’s service mesh uses - and it is why the mesh can claim “the data path never leaves the kernel” while still supporting userspace features: the control plane is userspace, the hot path is not.

8.5 SK_SKB: The Receive Side and the Full Proxy

The receive-side twin, SK_SKB, runs on data received by a socket in the map and can redirect that data to a peer socket (bpf_sk_redirect_map). Put SK_MSG and SK_SKB together and you have a complete in-kernel proxy: sends on socket A redirect to B; receives on B redirect (or pass) to A’s userspace if L7 handling is needed. The verdict semantics mirror SK_MSG: SK_SKB_PASS / SK_SKB_REDIRECT.

The direction flag is where the subtlety lives. BPF_F_INGRESS makes the redirected data land on the receive path of the target socket (so the target’s recv sees it); without it, the data goes to the target’s send path (for chaining). Getting the flag wrong is the classic sockmap bug: the bytes are redirected, the packet disappears into a queue nobody reads, and the connection hangs. Chapter 14’s testing chapter returns to exactly this class of bug with a reproduction recipe.

8.6 What sockmap Buys and What It Costs

The honest balance sheet, because every chapter in this book ends with one:

• Buys: no userspace round trip per message, no copy into userspace buffers, no proxy process on the hot path, kernel-managed flow control between the two sockets (a slow reader still throttles the writer - the receive queue semantics of Chapter 7 apply unchanged).
• Costs: the control plane must manage socket lifetime (a socket in a map is pinned by the map - closing the fd does not close it until it is removed), the map lookup per message, and the loss of userspace processing: any L7 logic (parsing, rewriting, auth) must happen either in the BPF program or back in userspace, which means the SK_MSG pass / redirect decision is a policy decision the control plane must set up correctly in advance.

The numbers that matter (measured in Chapter 13): an in-kernel sockmap redirect saves the two syscalls and the copy of a proxy round trip - roughly a 2-10x reduction in per-message latency for small messages, and a corresponding reduction in the CPU cost per connection. That is why Cilium uses sockmap for service routing and the mesh data path, and why the capstone builds its mesh on exactly this mechanism.

Hands-On Lab

# 1. Build the kernel side and run the control plane.
cd code/ch08_sockmap/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && sudo cargo run --release

# 2. Connect a client; the kernel moves the bytes from now on.
nc 127.0.0.1 8080 &

# 3. Prove no userspace proxy is in the path: the control plane process
#    shows zero read()/write() syscalls per message.
sudo strace -c -p <pid-of-userspace>     # after the handshake: quiet
sudo bpftool map dump name CONNS         # both sockets under one key

The comparison experiment: run the same load through a userspace proxy (a plain nc | nc pipe) and watch strace count a read+write per byte for both directions. That syscall+copy cost is what sockmap removes.

Summary

• sockmap/sockhash are BPF maps whose values are sockets; programs attach to the map and run on data sent to (SK_MSG) or received from (SK_SKB) sockets in it.
• The program’s verdict is PASS or REDIRECT, delivered by bpf_msg_redirect_hash / bpf_sk_redirect_map - a map-mediated handoff between sockets, entirely in the kernel.
• The control plane owns the sockets: it accepts, dials, and inserts both sides of a connection under keys; the kernel then moves the bytes.
• BPF_F_INGRESS chooses receive-path delivery on the target; the wrong flag produces a silent hang - the classic sockmap bug.
• The balance: no round trip and no copy against control-plane socket management and a per-message lookup - the trade that makes sockmap the data path of Cilium’s mesh.

Next: Chapter 9 moves the hook from the socket to the cgroup - where entire groups of sockets are filtered, counted, and rate-limited at once.

Chapter 9: cgroup eBPF - Socket Filtering & Resource Control

“The cgroup is where the kernel learns which process is which. Put an eBPF program there and it learns which group is which - and can govern every socket they open, connect, or use.”

Sockmap (Chapter 8) governs sockets one connection at a time. This chapter governs them by the container: the cgroup hooks, attached to a cgroup (v2, in modern kernels) and running on every socket operation made by any process in that group. Socket filters that see every packet of every socket in the group; connect/accept hooks that can rewrite addresses before the connection exists; sockopt hooks that police setsockopt; and the cgroup bandwidth controller that rate-limits the group’s traffic. This is the hook family that makes per-pod network policy possible - and the last piece of the socket-layer story before Part IV turns to Kubernetes, where all of it is applied.

9.1 The cgroup: The Kernel’s Grouping Primitive

A cgroup (control group) is the kernel’s mechanism for grouping processes and governing them as a unit: CPU shares, memory limits, and - since cgroup v2 and BPF - network behaviour. Every process belongs to exactly one cgroup in the hierarchy; in Kubernetes, the container runtime places each pod in its own cgroup (that is how kubectl knows what to kill, and how the OOM killer knows what to blame). For eBPF, the cgroup is a scope: attach a program to the cgroup’s socket hooks and it runs for every socket created or used by any process in that group.

The attachment mechanism is the same link-based one as everything else (Chapter 3): the userspace side opens the cgroup by path (/sys/fs/cgroup/...), attaches the program with a type (BPF_CGROUP_INET_INGRESS, etc.), and the kernel invokes the program at the matching socket operation.

9.2 The Hook Family, By Socket Lifecycle

A socket lives through stages, and cgroup eBPF has a hook at almost every stage:

For this book’s purposes - and for Cilium’s - the two that matter most are CgroupSkb (the per-group packet filter, which is how pod network policy is enforced on the socket layer) and CgroupSockAddr (which is how Cilium’s socket-level load balancing rewrites service addresses at connect() time).

9.3 CgroupSkb: The Group’s Packet Filter

The CgroupSkb program is the cgroup analogue of the socket filter of Chapter 6, but at group scope: one program, every packet of every socket in the group, ingress and egress. The verdict is the familiar one - return 1 to allow, 0 to drop - and the program receives the packet plus the socket’s identity through the context.

Source: code/ch09_cgroup_hooks/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch09_cgroup_hooks/kernel/src/main.rs
// Pod-level egress filtering, cgroup style: policy map, per-CPU counter,
// ring events for denials. Fail-open: a missing entry allows (Ch.9.3).
#![no_std]
#![no_main]

mod parse;

use aya_bpf::{
    macros::{cgroup_skb, map},
    maps::{HashMap, PerCpuArray, RingBuf},
    programs::CgroupSkbContext,
};

const MAX_POLICY: u32 = 1024;
const XDP_VERDICT_ALLOW: i32 = 1;

// The policy: key = dst ip, value = allow (1) / deny (0).
#[map]
static POLICY: HashMap<u32, u8> = HashMap::with_max_entries(MAX_POLICY, 0);

// Hot counter: per-CPU slots, no contention.
#[map]
static PKTS: PerCpuArray<u64> = PerCpuArray::with_max_entries(8, 0);

#[repr(C)]
pub struct DenyEvent {
    pub dst_ip: u32,
    pub ts: u64,
}
#[map]
static DENIES: RingBuf = RingBuf::with_max_entries(2048, 0);

#[cgroup_skb]
pub fn egress_filter(ctx: CgroupSkbContext) -> i32 {
    let Some(start) = ctx.data() else {
        return XDP_VERDICT_ALLOW; // no frame: allow, the stack decides
    };
    let frame = unsafe { parse::frame(start, ctx.data_end()) };

    if let Some(slot) = PKTS.get_ptr_mut(0) {
        unsafe { *slot += 1 };
    }

    // Fail-open: allow anything not explicitly denied.
    let Some(dst_ip) = parse::dst_ip(frame) else {
        return XDP_VERDICT_ALLOW;
    };
    match POLICY.get(&dst_ip) {
        Some(parse::ALLOW) => XDP_VERDICT_ALLOW,
        Some(_) => {
            emit_deny(dst_ip);
            0 // drop
        }
        None => XDP_VERDICT_ALLOW,
    }
}

fn emit_deny(dst_ip: u32) {
    let Ok(entry) = DENIES.reserve::<DenyEvent>(0) else {
        return; // ring full: the drop already happened, skip the event
    };
    unsafe {
        entry.write(DenyEvent {
            dst_ip,
            ts: aya_bpf::helpers::bpf_ktime_get_ns(),
        });
    }
    entry.submit(0);
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

The design choices are the book’s standing rules applied at group scope: bounds-checked parsing, per-CPU counters on the hot path, the ring buffer for events, and fail-open policy (a missing entry allows, never blocks). The one new idea is the scope: the same program, attached to a pod’s cgroup, governs every socket that pod owns - which is exactly the semantic Kubernetes network policy needs (Chapter 12 shows Cilium’s implementation).

9.4 CgroupSockAddr: Rewriting Connections at connect()

The CgroupSockAddr hook runs inside connect() (and bind()), before the connection exists, and it can rewrite the address the kernel will use. This is how socket-level load balancing works: the program sees “connecting to service 10.96.0.10:80” and rewrites it to “connecting to backend 10.1.2.3:8080” - the kernel then opens the real connection, and no packet-level NAT or proxy is needed at all. The rewrites are recorded in a map so that replies and subsequent packets can be correlated.

The difference from the tc/NAT approach (Chapter 6) is fundamental: packet-level NAT rewrites packets that already exist; socket-level redirection rewrites the connection before it exists. The former runs on the data path and must see both directions; the latter runs once, at connect, and the kernel’s own socket plumbing does the rest. Cilium uses both - CgroupSockAddr for new connections, packet-level eBPF for the rest - and Part IV returns to the split.

9.5 Bandwidth: The Group as a Rate-Limited Entity

The final cgroup capability in this book is bandwidth control: eBPF programs attached to the cgroup that implement token-bucket shaping for the group’s traffic, and the kernel’s max/high rate limits exposed through the cgroup filesystem. The token bucket is the classic shape - a bucket that refills at the configured rate, drains per packet, and drops (egress) or delays packets when empty - implemented in a BPF program with a per-cgroup map for the bucket state. The design lesson is the one that repeats through this book: the state (tokens, timestamps) lives in maps, the decision (allow/delay/drop) is per-packet, and the policy (rates) comes from userspace.

9.6 Attaching to a Pod: The Kubernetes Connection

In Kubernetes, each pod’s processes live in a cgroup, so the attach is: find the pod’s cgroup path (via the container runtime or /sys/fs/cgroup), attach the program with the right type, and the program now governs that pod. Aya’s CgroupSkb and friends attach to an open cgroup fd; the userspace side resolves the path, attaches, and fills the policy maps:

Source: code/ch09_cgroup_hooks/userspace/src/main.rs

// code/ch09_cgroup_hooks/userspace/src/main.rs
// Attach the egress filter to a pod's cgroup and fill its policy.
use aya::{maps::HashMap, programs::CgroupSkb, Ebpf};
use std::error::Error;
use std::fs::File;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const DEFAULT_CGROUP: &str = "/sys/fs/cgroup/kubepods/pod-demo";
const ALLOWED_DST: u32 = 0x0a00_0001; // 10.0.0.1
const ALLOW: u8 = 1;

fn main() -> Result<(), Box<dyn Error>> {
    let cgroup_path = std::env::args()
        .nth(1)
        .unwrap_or_else(|| DEFAULT_CGROUP.to_string());
    let cgroup = File::open(&cgroup_path)?;

    let mut ebpf = Ebpf::load(KERNEL_ELF)?;
    let prog: &mut CgroupSkb = ebpf.program_mut("egress_filter")?.try_into()?;
    prog.load()?;
    prog.attach(cgroup.try_clone()?, aya::programs::CgroupSkbAttachType::Egress)?;
    println!("egress filter attached to {cgroup_path}");

    let mut policy: HashMap<_, u32, u8> = ebpf.map_mut("POLICY")?.try_into()?;
    policy.insert(ALLOWED_DST, ALLOW, 0)?;
    println!("policy: 10.0.0.1 allowed, everything else passes (fail-open)");
    Ok(())
}

That is the complete per-pod data path: the cgroup scopes the program, the map carries the policy, the ring reports the denials. Chapter 12 shows Cilium scaling this from one pod to a whole cluster - but the primitive is exactly the one you just attached.

Hands-On Lab

# 1. Create a scratch cgroup and run a shell inside it.
sudo mkdir -p /sys/fs/cgroup/demo && echo $$ | sudo tee /sys/fs/cgroup/demo/cgroup.procs

# 2. Attach the egress filter to it and fill the policy.
cd code/ch09_cgroup_hooks/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && sudo cargo run --release -- /sys/fs/cgroup/demo

# 3. Prove the scope: traffic from the cgroup is filtered, traffic
#    outside it is not.
curl -s http://10.0.0.1/ >/dev/null    # inside the cgroup: allowed
# add a deny rule for 10.0.0.2, then curl it from inside: dropped + event
sudo bpftool map dump name DENIES

The fail-open property is worth testing on purpose: delete the policy map entry and confirm the pod’s traffic still flows. A policy engine that black-holes on an empty table is a network incident waiting to happen.

Summary

• cgroups are the kernel’s grouping primitive and the scope for a family of socket hooks: one program governs every socket of every process in the group.
• The hooks map to the socket lifecycle: CgroupSkb filters packets, CgroupSockAddr rewrites connects, CgroupSockopt polices options.
• CgroupSkb is the per-group packet filter: bounds-checked parse, policy map, per-CPU counters, ring events, fail-open.
• CgroupSockAddr rewrites the connection before it exists - the socket-level LB mechanism, distinct from packet-level NAT.
• Bandwidth control is a token bucket in BPF maps, with rates from userspace.
• Attaching to a pod is attaching to its cgroup: the primitive behind per-pod network policy, ready to be scaled by Part IV.

Next: Part IV steps out of the single host and into the cluster - Chapter 10 is the Kubernetes networking model: CNI, kube-proxy, and the service abstraction eBPF was built to replace.

Chapter 10: The Kubernetes Networking Model - CNI, kube-proxy & Services

“Kubernetes networking is a promise the platform keeps with a thousand little machines: every pod can reach every pod, everywhere, on a flat network - and the promise is kept by CNI plugins, iptables chains, and increasingly, eBPF.”

Everything in Parts I-III ran on one host. This chapter opens the cluster: what Kubernetes promises about networking, how the CNI layer keeps that promise at pod level, how Services add a stable name-and-port abstraction, and how kube-proxy implements it with iptables and IPVS - the machinery eBPF was built to replace. If Part IV has a thesis, it is this: Kubernetes networking is a translation problem - translating the cluster’s logical model (pods, services, policies) into the kernel’s physical model (interfaces, IP addresses, NAT tables) - and every layer of that translation is a place where eBPF can be faster and more correct than the machinery it replaces.

10.1 The Four Promises

The Kubernetes networking model is famously short - four rules, from the documentation, with no implementation details:

1. Every pod gets its own IP address, cluster-wide, unique.
2. Every pod can reach every other pod at that IP, on any node, without NAT in between.
3. Every pod can reach every service (and every node, and the outside world).
4. The pod’s view of the network is flat: it does not know about nodes, tunnels, or the network that carries its traffic.

The consequences are what make the model hard. Because pods move between nodes (rescheduling, scaling), their IPs must be portable: the network must carry pod-to-pod traffic across node boundaries. Because pods can be created and destroyed at will, address allocation must be dynamic. And because rule 4 forbids NAT between pods, the cluster needs an overlay or a flat L2/L3 fabric - which is where CNI comes in.

10.2 CNI: The Interface Between Runtime and Network

CNI (Container Network Interface) is the plugin contract between the container runtime and a network implementation. The runtime calls the CNI plugin binary with a JSON spec on stdin (“add this container to the network”); the plugin must create the pod’s network: a veth pair (one end in the pod’s network namespace, one end on the host), an IP address from its allocation pool, routes, and any encapsulation. The pod’s network namespace is then one hop from the host - which is why the pod’s view of the network (rule 4) is preserved: the pod sees its veth, the host sees the other end, and everything beyond is the plugin’s business.

The two families of CNI implementations map to the two models from Chapter 1:

• Routing-based (Calico, Cilium in routing mode): the plugin assigns real IPs and programs routes on the host (and the fabric), so pod-to-pod traffic is routed like any IP traffic. Fast, no encapsulation tax, but the fabric must be route-aware.
• Overlay-based (Flannel, Weave, Cilium in VXLAN mode): the plugin wraps pod traffic in an encapsulation (VXLAN/Geneve) and tunnels it across the cluster. Works on any fabric, at the price of header overhead and a per-packet encapsulation cost.

The packet path inside a node is the Chapter 1 path with names: pod veth -> host veth -> tc hooks -> routing -> (encapsulation if overlay) -> node NIC. Every hook in this book is on that path, and every CNI that uses eBPF (Cilium is the famous one) is putting the Chapter 5-9 programs exactly there.

10.3 Services: The Stable Abstraction

Pods die; the service stays. A Service is a stable virtual IP (ClusterIP) and port that fronts a set of pods selected by label; when a client connects to the service, the cluster routes the connection to one of the backing pods. The routing is the whole story: the service does not exist as a network object. It is a rule - “connections to 10.96.0.10:80 go to one of {pod-a, pod-b, pod-c}” - that some component must turn into packet-level decisions on every node.

That component has been kube-proxy, and its history is the history of the networking problem eBPF solves:

• Userspace mode (the original): kube-proxy owned the service IP and proxied connections in userspace. Correct, slow, and now ancient.
• iptables mode (the default for a decade): kube-proxy programs the kernel’s netfilter with a chain per service and per pod. Each packet to a service walks a linear chain of rules; the chain for a cluster with many services is long, and the first packet of a connection pays for all of it. Correct, and the source of “why is my cluster’s first-packet latency bad” mysteries.
• IPVS mode: kube-proxy programs the kernel’s IPVS (IP Virtual Server) tables instead - a hash-based LB engine - replacing the linear walk with a lookup. Better, but still netfilter, still a userspace agent reconciling state, and still per-node policy that must be kept in sync.

The recurring cost is translation: a userspace agent (kube-proxy) watches the API server, computes rules, and pushes them into the kernel through a slow, indirect mechanism. Every watch event, every resync, every partial failure is a chance for the kernel’s view to lag the cluster’s truth. That is the job eBPF takes over in Chapter 11.

10.4 The iptables Walk: What a Packet Actually Does

To appreciate the replacement, walk what a packet does today under kube-proxy in iptables mode:

client pod -> service ClusterIP 10.96.0.10:80
  -> OUTPUT chain (the pod's own node)
  -> KUBE-SERVICES: match 10.96.0.10
  -> KUBE-SVC-XXXX (one chain per service): random selection among pods
     -> KUBE-SEP-AAAA (endpoint A): DNAT to pod-a:8080
  -> the packet is now addressed to pod-a
  -> FORWARD chain, then the CNI's data path (veth, routes, overlay)
  -> pod-a's node, pod-a's veth, pod-a's process

Every service adds a chain; every endpoint adds a chain; the -m statistic --probability randomisation is a chain decision. The cost model is the one Chapter 6 established: each rule is a linear comparison, and the first packet of every connection walks from the top. With hundreds of services, the first-packet walk is the tail latency you can measure. The eBPF replacement (next chapter) is the same decision - “10.96.0.10:80 goes to pod-a” - but as a hash lookup in a BPF map: one lookup, O(1), before the packet ever reaches netfilter.

10.5 DNS and the Service Ecosystem

One more piece of the model before Cilium: DNS. Pods resolve service names through the cluster’s CoreDNS, and the resolution is the control plane of service discovery - it happens rarely. The data plane - routing connections to the resolved IP - happens per packet. The separation matters because it is the same separation this book has drilled since Chapter 3: control plane (what resolves, what changes) and data plane (what executes per packet). Cilium’s innovation is to compress more of the control plane’s output into BPF maps so the data plane does less work, and to observe the data plane (Hubble, Chapter 12) so the control plane’s mistakes are visible.

10.6 Service Types: ClusterIP, NodePort, LoadBalancer, ExternalName

The Service abstraction has four flavours, and each one changes the translation problem - which is why the eBPF replacement must handle all of them, not just ClusterIP:

• ClusterIP: the default. A virtual IP reachable only inside the cluster. The rule is “connections to 10.96.0.10:80 go to a backend pod”.
• NodePort: a port opened on every node (e.g. 30080), forwarded to the ClusterIP, then to a backend. The translation problem gains one hop: node -> ClusterIP -> pod.
• LoadBalancer: the cloud provider provisions an external LB (an ELB/NLB, or MetalLB in bare metal) whose backends are the nodes; traffic arrives as NodePort traffic. One more hop in the provider’s control plane, the same kernel machinery underneath.
• ExternalName: no IP at all - a DNS CNAME. The data plane never sees it; the translation is pure DNS.

The engineering consequence: a kube-proxy replacement must implement the whole family - external traffic policy (ExternalTrafficPolicy: Local preserves the client IP but pins the service to one node), session affinity (stickiness by client IP), and health checking of backends. When Chapter 11 shows Cilium’s service maps, remember that each service type is a different key shape in those maps - ClusterIP keys, NodePort keys, and the external-TrafficPolicy decisions are map entries, not code paths.

10.7 What You Can Do Today, With What You Know

Before moving on, notice that you already know how to build a kube-proxy replacement from the previous five chapters:

• Service routing at the socket layer: CgroupSockAddr (Chapter 9) rewrites the connect to a backend pod - exactly the socket-level LB of Chapter 9.4.
• Service routing at the packet layer: a tc or XDP program (Chapters 5-6) does the ClusterIP->pod DNAT with a hash map keyed by the service tuple - exactly the iptables walk, but O(1).
• Session affinity: a per-client hash in the map (Chapter 4’s HASH with a client-IP key) instead of -m statistic randomness.
• Observability: the ring buffers (Chapter 4) and tracepoints (Chapter 7) report who connected to what - the beginnings of Hubble.

The hands-on verification that makes the model concrete: on any cluster with kube-proxy in iptables mode, iptables -t nat -L KUBE-SERVICES -n shows the chain walk of Section 10.4 - one line per service, one chain per endpoint, and the statistic rules that randomise. Count the lines, then time the first packet of a new connection (hping3 -S or nc -vz against a fresh ClusterIP). That measured walk is what Chapter 11 replaces with a hash lookup.

Chapter 11 shows the production version of exactly this, from the project that made eBPF famous: Cilium.

Hands-On Lab

# 1. On any cluster with kube-proxy in iptables mode, see the walk.
kubectl create deployment web --image=nginx
kubectl expose deployment web --port=80 --target-port=80
iptables -t nat -L KUBE-SERVICES -n     # one chain per service
iptables -t nat -L KUBE-SVC-XXXXXX -n   # one chain per endpoint

# 2. Measure the first-packet cost the chains create.
kubectl run client --image=nicolaka/netshoot -- sleep 3600
kubectl exec client -- sh -c 'time curl -s http://web >/dev/null'   # cold
kubectl exec client -- sh -c 'time curl -s http://web >/dev/null'   # warm

The cold-vs-warm gap is the chain walk; Chapter 11 replaces it with a map lookup whose cost does not grow with the number of services.

Summary

• The Kubernetes networking model is four promises: unique pod IPs, full pod-to-pod reachability without NAT, pod-to-service reachability, and a flat pod view of the network.
• CNI is the plugin contract that builds the pod’s network (veth, IP, routes) - routing-based or overlay-based, both running on the Chapter 1 packet path.
• Services are rules, not network objects: a virtual IP that some component must translate into per-packet decisions.
• kube-proxy has done that with userspace, iptables (linear chains - the classic tail-latency story), and IPVS - all userspace agents reconciling kernel state.
• The eBPF thesis: the same decisions as hash lookups in BPF maps, made in the kernel, before netfilter - the subject of Chapter 11.

Next: Chapter 11 is the main event of Part IV - Cilium, the eBPF data plane that replaced kube-proxy, chain by chain, map by map.

Chapter 11: Cilium - The eBPF Data Plane for Kubernetes

“Cilium is what happens when the chapters of this book get shipped: XDP and tc programs on every node, sockmap in the socket layer, cgroup hooks on every pod, and a control plane that compiles the cluster’s intent into BPF maps - no iptables, no kube-proxy, no userspace proxy on the hot path.”

Chapter 10 described the problem: a userspace agent translating the cluster’s logical model into kernel rules through iptables. This chapter describes the answer that made eBPF famous: Cilium, the CNI and network data plane built entirely on the mechanisms of Parts I-III. By the end you will be able to read a Cilium architecture diagram the way you read a packet path - as a set of BPF programs, maps, and hooks you already know - and you will understand why the project’s claim “no iptables, no kube-proxy” is a statement about where the decisions are made, not a marketing slogan.

11.1 The Architecture: Agent, Datapath, and the API Server

Cilium’s components map cleanly onto the two-plane discipline of this book:

• The agent (cilium-agent, a daemonset on every node) is the control plane: it watches the Kubernetes API server (services, endpoints, pods, policies), and compiles those objects into BPF programs and map entries. It loads the programs, attaches them to the node’s interfaces and cgroups, and updates the maps as the cluster changes. This is the Aya user in Chapter 3, scaled to a cluster.
• The datapath is the set of BPF programs the agent attaches: an XDP program for the node’s external interface, tc programs on the veth pairs of every pod, sockmap programs in the socket layer, cgroup hooks on every pod’s cgroup. These are the programs of Chapters 5, 6, 8 and 9, compiled from C (with Rust in the tooling increasingly) into the same ELF objects the Aya loader understands.
• The API server is the source of truth; the agent is the translator; the maps are the compiled output. When a Service is created, the agent does not add an iptables chain - it inserts an entry into the BPF service map, and the data path changes on the next packet.

The crucial difference from kube-proxy is the unit of work. kube-proxy programs netfilter, a state machine with its own view of the world; Cilium programs BPF maps, which are just data the kernel consults per packet. Updating a map is atomic, cheap, and immediately visible - there is no “wait for the iptables resync”, no half-applied chain, no per-node drift beyond the agent’s own watch latency.

11.2 The Service Data Path: From ClusterIP to Pod

Follow a connection to a Service under Cilium, using the hook ladder of Parts I-III:

1. Socket layer (new connections): a pod’s connect() to 10.96.0.10:80 hits the CgroupSockAddr program (Chapter 9), which looks up the service in a BPF map and rewrites the address to a backend pod. The kernel opens the real connection; no packet ever carries the ClusterIP, and no NAT runs on the data path.
2. Packet layer (everything else): traffic that arrives without a socket-level rewrite - forwarded traffic, traffic from outside, UDP - is handled by the tc programs on the veths and the XDP program on the node’s NIC: parse the tuple, hash it (consistent hashing for stickiness, Maglev-style for even spread), look up the backend in the service map, DNAT/encapsulate as needed, redirect.
3. Conntrack: Cilium maintains connection state in BPF maps - its own conntrack (CT), keyed by the 4-tuple, recording the mapping between the client’s view (ClusterIP) and the backend’s view (pod IP) so that reply packets are translated back without recomputation. This is Chapter 7’s netfilter conntrack, reimplemented in the map vocabulary of Chapter 4 - and it is why Cilium can claim per-node conntrack with no dependence on nf_conntrack.

The result is the Chapter 10 iptables walk reduced to one hash lookup, one map entry, one conntrack insert - before the packet reaches netfilter. The first-packet tail latency that iptables chains produced (Chapter 10.4) is replaced by the deterministic cost of a map lookup.

11.3 Why “No kube-proxy” Matters: The Numbers

The performance claims deserve the honest treatment of Chapter 13, but the shape of the argument is clear from the architecture alone:

• PPS capacity: the data path is XDP + tc + sockmap - the hooks this book measured as ~100 ns (drop), ~0.5-1 us (redirect), ~1-5 us (tc end-to-end). iptables’ linear chain walk on first packets is the difference between hundreds of thousands of new connections per second and millions, and between Mpps-scale and tens-of-Mpps-scale forwarding.
• First-packet latency: a map lookup vs a chain walk - the difference between microseconds and milliseconds on cold services, which is the classic “why is the first request slow” story in iptables clusters.
• CPU: no kube-proxy process on every node, no netfilter traversal on every packet, no conntrack table in the netfilter subsystem - the kernel does less per packet because the decision is a lookup, not a traversal.

The honest counterpoints (Cilium’s docs say them too): the agent is more complex than kube-proxy (more moving parts in the control plane), the map tables must be sized and evicted (Chapter 4’s LRU_HASH discipline), and the kernel must be recent enough for the features used (BTF, the modern hooks - the platform requirements of the foreword).

11.4 The Datapath Programs, Mapped to This Book

Cilium’s source tree (bpf/ directory) is the best real-world reading companion to Parts I-III. The mapping is direct:

Read any of them with the Chapter 5 parse-bounds-lookup pattern in mind and they stop being “the Cilium codebase” and become “a bigger version of the programs in this book”. That is the point of learning the primitives before the product.

11.5 Running Cilium: The Hands-On Minimum

Cilium is the one component of this book you should run before the capstone, because the capstone runs inside it. The minimal path:

# A kind cluster with Cilium as the CNI (the capstone's environment).
kind create cluster --name ebpf-book
helm repo add cilium https://helm.cilium.io
helm install cilium cilium/cilium \
  --namespace kube-system \
  --set kubeProxyReplacement=true \
  --set hostServices.enabled=true

# Verify the data path is eBPF, not iptables.
cilium status
kubectl -n kube-system get pods -l k8s-app=cilium

# The moment of recognition: watch the BPF programs the agent loaded.
bpftool prog list | grep -E 'xdp|sched_cls|sock'   # Chapter 14's tool

When bpftool prog list shows Cilium’s sched_cls programs attached to veth pairs, you are looking at Chapter 6’s hook with a daemonset in front of it. When cilium status reports KubeProxyReplacement: True, you are looking at Chapter 10’s problem solved by Chapter 4’s maps. The capstone (Chapter 16) builds a miniature of exactly this on the same cluster.

11.6 The Datapath Design Review: What to Read, and Why

The moment the model meets the code, use the source tree as a textbook. The bpf/ directory is organised exactly like Part II of this book, and reading it with the Chapter 5-9 patterns in mind turns “the Cilium codebase” into “a bigger version of the programs I have written”:

• bpf_xdp.c - the XDP program of Chapter 5 at node scale: parse the frame with data/data_end, check the tuple against a service map, XDP_REDIRECT. Find the bounds discipline you practised in Chapter 5, and watch what a production program adds: frag handling, IPv4/IPv6, and the encapsulation decisions.
• bpf_lxc.c - the tc program of Chapter 6 on every veth: the per-endpoint policy lookup (the identity pair of Chapter 12), the conntrack insert, the L4 check. This is Chapter 9’s cgroup filter generalised from “this pod” to “any endpoint, identified by a 24-bit identity”.
• bpf_sock.c - the cgroup hooks of Chapter 9: the socket-level service rewrite you studied in Section 11.2, in production form.
• bpf_sockmap.c - the sockmap of Chapter 8, with the same fail-open discipline around bpf_msg_redirect_map.

The review habit that pays off: for each program, answer the three questions this book has trained you to ask - which hook? which maps? what is the fail-open path? The answers are visible in the source within a few hundred lines, because Cilium is written with the same discipline the CODING_STANDARDS of this book enforce: bounds first, maps for state, actions as the only output.

11.7 ClusterMesh: The Multi-Cluster Extension

The last architectural note before Chapter 12: ClusterMesh extends the same model across clusters. Each cluster runs its own agent and datapath, and a control plane shares service metadata (identities, endpoints, policies) between them - so a service in cluster A can have backends in cluster B, with the routing still decided by the same BPF maps, just with tunnel endpoints for the inter-cluster leg. The design consequence is the one this book has repeated at every scale: the data plane never learns a new trick for multi-cluster; the control plane simply writes different map entries (a backend IP in cluster B instead of cluster A). Whatever the cluster looks like, the per-packet work stays “parse, look up, act” - and the capstone in Chapter 16 is a miniature of that statement.

Hands-On Lab

# 1. The cluster from the chapter.
kind create cluster --name ebpf-book
helm repo add cilium https://helm.cilium.io
helm install cilium cilium/cilium --namespace kube-system \
  --set kubeProxyReplacement=true --set hostServices.enabled=true

# 2. The moment of recognition: the hooks of this book, with a daemonset.
cilium status                      # KubeProxyReplacement: True
kubectl -n kube-system get pods -l k8s-app=cilium
sudo bpftool prog list | grep -E 'xdp|sched_cls|sock'

# 3. Watch a Service become a map entry.
kubectl expose deployment web --port=80
sudo bpftool map dump | grep -A2 80   # the ClusterIP key appears

Every line of bpftool prog list output corresponds to a chapter in this book: xdp is Chapter 5, sched_cls Chapter 6, sock Chapters 8-9.

Summary

• Cilium is the agent (control plane) + BPF programs (data plane) architecture: the agent watches the API server and compiles cluster intent into BPF maps; the programs execute per packet.
• The service path uses the whole hook ladder: CgroupSockAddr for new connections, XDP/tc for packet traffic, sockmap for the mesh, BPF-map conntrack for replies - the iptables walk replaced by hash lookups.
• “No kube-proxy” means the decisions moved from netfilter chains to BPF maps: first-packet latency and PPS capacity change because the unit of work changed, not because of magic.
• The datapath programs map 1:1 to this book’s hooks - reading Cilium’s bpf/ is reading a larger Chapter 5-9.
• Run it: kind + helm install cilium, then bpftool prog list is the moment the model meets the machine.

Next: Chapter 12 layers the rest of the cloud story on the data path - the Cilium service mesh, network policies, mTLS, and Hubble observability.

Chapter 12: Cilium Service Mesh, Security & Hubble

“A service mesh is a decision about who is allowed to talk to whom, and how - executed without a proxy in the path. Cilium’s answer is: the policy is a map, the execution is the kernel, and the proof is in the flows.”

Chapter 11 built the data plane. This chapter builds the policy layer on top of it: how Cilium turns the cluster’s security intent into the mechanisms of Chapters 4-9 - network policies compiled into map entries and executed by the tc/sockmap programs, mTLS between pods without a userspace proxy for the data path, the L7 proxy that appears only when L7 rules demand it, and Hubble, the observability layer that turns the ring buffers of Chapter 4 into the cluster’s flow log. When you finish this chapter you will understand the full arc of the book: from one packet on one NIC, to the security policy of an entire cluster, through the same primitives.

12.1 Network Policy: Intent Compiled into Maps

A Kubernetes NetworkPolicy says “pods with label app=api may receive from pods with label app=frontend on port 443”. Cilium extends this with its own CiliumNetworkPolicy (L3-L7, CIDRs, FQDNs, TLS-aware). The architecture is the one from Chapter 11: the policy is data, not code. When a policy lands, the agent:

1. Resolves the labels to identities. Every pod gets an identity (a 24-bit number) derived from its labels. Identities are the currency of the policy engine: the map key is an identity pair, not a pod pair, which is what makes policy survive pod churn - a new pod with the same labels inherits the policy without a new rule.
2. Compiles the policy into map entries. The endpoint’s policy maps (LXC_ID -> allowed identity -> allowed L4/L7) are updated. The tc program on the pod’s veth (Chapter 6) consults them per packet.
3. Enforces at both ends. Cilium enforces egress policy at the source pod and ingress at the destination pod - two independent checks, so a compromised node cannot bypass one end’s policy.

The per-packet cost is what Chapters 4-6 prepared you for: a tuple -> identity lookup, a policy map lookup, an L4 check - all O(1) map operations. There is no rule engine on the data path. This is the architectural statement that separates eBPF policy from iptables policy: iptables evaluates a list of rules per packet; Cilium evaluates a set of maps per packet. The worst case of the former grows with policy size; the worst case of the latter does not.

12.2 The L7 Story: Proxy Only When You Ask For It

L4 policy (IP, port, identity) is free with the maps above. L7 policy (HTTP paths, Kafka topics, gRPC methods) needs to see the payload, and payload parsing is not a verifier-friendly activity. Cilium’s answer is the honest hybrid this book has been teaching since Chapter 8:

• The default data path stays in the kernel. L4 policy, routing, LB - all in BPF, no proxy.
• An Envoy-based proxy (cilium-envoy) is deployed per node, and a packet whose flow matches an L7 rule is redirected to it - but only that flow, and only at the socket level (sockmap, Chapter 8), so the redirect is a map lookup, not a packet-level hairpin.
• The policy engine decides which flows need L7 by inspecting the policy maps before the data path decides.

The design principle to internalise: the proxy is a fallback for the cases the kernel cannot cheaply handle, not the default path. L7 rules are the exception in most clusters; paying for them on every packet is what the older service meshes did, and it is precisely the cost Cilium removes by making the proxy opt-in per flow.

12.3 mTLS Without the Proxy Tax

Mutual TLS between pods is the security baseline of a mesh: every connection is authenticated and encrypted in both directions. The classic mesh does this in a sidecar proxy - which means every byte crosses the proxy, and the mesh’s latency and CPU story is dominated by the proxy. Cilium’s approach splits the problem the same way the data path does:

• The handshake and certificate lifecycle are control plane: the agent manages certificates (via the cert-manager integration or a CA), and the first handshake of a connection can involve the agent/Envoy for the certificate exchange.
• The steady-state encryption is kernel-level: the kernel’s IPsec (XFRM) is programmed with the negotiated keys, so the data path is encrypt/decrypt in the kernel’s crypto stack - no userspace proxy in the byte path, with the kernel’s crypto acceleration (AES-NI, etc.) applying.

The honest trade: kernel IPsec gives you per-packet performance and no proxy CPU, at the cost of features the proxies offer (protocol-level inspection, retries, traffic shaping) - which is exactly why the L7 proxy of Section 12.2 coexists with it, and why the policy decides which path a flow takes. The mesh is a policy-compiled routing decision, like everything else in this chapter.

12.4 Hubble: The Ring Buffer, Scaled to a Cluster

Hubble is Cilium’s observability layer, and for this book it is the satisfying payoff of Chapter 4’s ring buffer. The data path programs emit one event per significant flow transition - connection established, policy applied, packet dropped, reply seen - through the ring (or the older perf buffer) to the agent, which enriches the events with Kubernetes context (namespaces, labels, identities) and serves them through the Hubble API and UI. A flow entry looks like:

K8s Namespace: default   K8s Pod Name: frontend-7d8f5b
Source IP: 10.0.1.5      Destination IP: 10.0.2.9
Destination Service: api:443
Verdict: FORWARDED       Policy: allowed-by: {"l4":["443/TCP"]}

The design is the one you have used since Chapter 4: the kernel produces events, userspace enriches, humans query. The eBPF part is the cheap part (a ring write per event, amortised); the value is the enrichment, which is why Hubble’s API returns kubectl-shaped answers and not raw tracepoint dumps. When the capstone ships its own flow events through its own ring (Chapter 16), it is building a one-node Hubble.

12.5 The Policy You Can Write Today

The hands-on minimum that makes the chapter real:

# A policy that says: only pods labeled app=frontend may reach app=api on
# port 443, over mTLS, with HTTP path /healthz allowed at L7.
cat <<'EOF' | kubectl apply -f -
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: api-ingress
spec:
  endpointSelector:
    matchLabels: {app: api}
  ingress:
  - fromEndpoints:
    - matchLabels: {app: frontend}
    toPorts:
    - ports: [{port: "443", protocol: TCP}]
      rules:
        http:
        - method: GET
          path: "/healthz"
EOF

# Watch the policy become data: the endpoint's policy map is updated, and
# hubble shows the flows it allows.
cilium endpoint list
hubble observe --from-pod frontend --to-pod api

The two commands are the chapter in miniature: the policy is a YAML object until the agent compiles it; then it is a set of map entries, and the proof of enforcement is a flow log. That is the entire arc of this book, in two commands.

12.6 Where the Mesh Ends and Your Code Begins

The service mesh is the top of the stack this book climbs - but notice what it is not: it is not a separate system. It is the same hooks (Chapters 5-9), the same maps (Chapter 4), and the same two-plane discipline (control plane compiles, data plane executes) that you have been using since Chapter 3. When the capstone builds its mesh in Chapter 16, it will assemble: XDP LB (Chapter 5), sockmap redirect (Chapter 8), a policy map (Chapter 4), and a flow ring (Chapter 4) - and it will be a Cilium in miniature. The distance from “I wrote a packet filter” to “I understand a service mesh” is exactly the distance from Chapter 5 to this chapter - the primitives did not change; only the scale of the intent did.

Hands-On Lab

# 1. Apply the L4 policy from Section 12.5, then generate a matching flow.
kubectl apply -f api-ingress.yaml
kubectl run client --image=nicolaka/netshoot -- sleep 3600
kubectl exec client -- curl -s https://api:443/healthz

# 2. Watch the policy become data and the flow become a log.
cilium endpoint list               # policy enforcement state per pod
hubble observe --from-pod client   # the flow, verdict and allowed-by
kubectl exec client -- curl -s http://api:80/    # non-allowlisted: blocked
hubble observe --deny                             # the denial appears

The proof that policy is compiled, not interpreted: the per-packet cost does not change when you add the 10th or the 100th policy rule - it is a map lookup either way (Chapter 12.1).

Summary

• Network policy is compiled into map entries: identity pairs, not pod pairs, key the policy maps; enforcement happens at both ends of every connection, per packet, O(1).
• The L7 proxy is opt-in per flow: the default data path is BPF; only flows matching L7 rules are socket-redirected to per-node Envoy.
• mTLS splits the problem: certificate lifecycle in the control plane, steady-state encryption in kernel IPsec - no proxy in the byte path.
• Hubble is the ring buffer scaled to a cluster: kernel events, userspace enrichment, human queries - the observability pattern of Chapter 4, productionised.
• The mesh is not a separate system: it is the same hooks, maps, and two-plane discipline as everything before - the capstone proves it.

Next: Part V turns from features to operations. Chapter 13 is the performance methodology - the per-packet budget of Chapter 1, measured and spent well.

Chapter 13: Performance Engineering for eBPF

“An eBPF program does not have a performance problem. It has a budget - the cycles between one packet and the next - and the entire discipline is spending that budget where it matters, and proving where it went.”

Every chapter so far has quoted a budget: 4000 / R cycles per packet at R Mpps (Chapter 1), map lookups in tens of nanoseconds (Chapter 4), XDP drops in ~100 ns (Chapter 5). This chapter is where the budget becomes a methodology: how to measure an eBPF data path, how to spend the budget, and how to prove the result. It covers the per-packet cost model in detail, the verifier’s performance constraints, the map and hook choices that move the needle, the profiling tools that tell you where the cycles went, and the discipline of reproducible benchmarks (CODING STANDARDS rule 8: kernel, NIC, CPU, flags, method - all recorded).

13.1 The Budget, Expressed as a Methodology

The starting point is the per-packet cycle budget of Appendix B:

budget = (cores * frequency) / packet_rate        cycles per packet

At 4 GHz total and 5 Mpps, that is 800 cycles per packet. The methodology is: account for the budget before you write the program. The fixed costs of any network program are known: a map lookup (Chapter 4) is tens of nanoseconds; a bounds check is a few cycles; a helper call is a call plus whatever the helper does. If the design needs three lookups and a redirect at 5 Mpps, count the cycles first - and if the sum is over budget, change the design (fewer lookups, a per-CPU map, a different hook) before you write a line of BPF.

13.2 The Cost Table, Measured

The numbers below are the ones this book’s design reasoning uses (typical 6.x kernel, x86-64, measured with the tools of Section 13.5 - your kernel will differ slightly, which is why the benchmark method is mandatory):

The table’s shape is the book’s thesis in numbers: the map is cheap, the hook placement decides the order of magnitude, and the helper you call is where the variance hides. A program that looks up one per-CPU array and returns an action is a few tens of nanoseconds; a program that calls bpf_skb_store_bytes on every packet pays reallocation when the skb grows.

13.3 Spending the Budget: The Design Rules

The rules that emerge from the table, in order of leverage:

1. Pick the hook by the decision, not the fashion. The drop-easy / decide-late rule of Chapter 6 is a budget rule: XDP’s ~100 ns drop only exists because the stack never ran. If your decision needs conntrack, tc is cheaper than “XDP plus hand-rolled conntrack”.
2. Per-CPU maps for anything on the hot path. The difference between a contended HASH counter and a PERCPU_ARRAY counter is cache-line contention - the Chapter 10 false-sharing lesson of the C++ book, applied to the kernel. The per-CPU map is not an optimisation; it is the default.
3. One lookup beats two. The composite key of Chapter 4 - a service tuple as one u64 - is a budget rule: it halves the lookup cost of the Chapter 11 service path.
4. Batch or amortise the expensive parts. Ring events are cheap per event but the syscall to read them is not; userspace drains the ring in batches (Chapter 4’s iterator), and the kernel side writes only what the control plane actually needs. Observability is a budget line, not a free feature.
5. The verifier’s instruction budget is a design constraint. A program that needs 1 M instructions (the 6.x BPF_MAXINSNS) is a program that will not survive at Mpps. If the verifier’s instruction count grows, the design grew; move work to maps or to userspace, not to the program.

13.4 Where the Cycles Actually Go: Profiling

The budget says where cycles should go; profiling says where they do go. The toolchain, in order of leverage:

• perf: perf record on the workload, then inspect the BPF programs’ share. The BPF JIT symbols show up as bpf_prog_<id>_<name>; their weight tells you which program dominates.
• bpftool prog show and bpftool prog dump jited: the JIT output is the ground truth - how many native instructions the program became. A 200-instruction program with a 20-instruction hot loop is a different problem from a 2,000-instruction program.
• Map statistics: bpftool map show reports the number of lookups, updates, and (for LRU maps) evictions per second - the map’s contention and hit rate are visible without any instrumentation of your own.
• perf stat on the syscall side: for userspace, the batching discipline of Chapter 4 is measurable as syscalls per second.

The profiling loop is the same as the C++ book’s: hypothesise the hot instruction, measure, change, re-measure. The difference is that in eBPF the JIT output is small enough to read, and the map stats are accurate enough to trust - the profiling loop converges fast.

13.5 The Benchmark: Reproducible, Honest, Comparative

Source: code/ch13_performance/userspace/src/main.rs

// code/ch13_performance/userspace/src/main.rs
// The reproducible benchmark: pump packets, drain the ring, print the
// per-packet cost WITH its conditions (rule 8: kernel, method, result).
use std::fs;
use std::time::Instant;

const PACKETS: u64 = 1_000_000; // synthetic packets to pump

fn kernel_release() -> String {
    fs::read_to_string("/proc/sys/kernel/osrelease").unwrap_or_else(|_| "unknown".into())
}

fn main() {
    let start = Instant::now();

    // The real benchmark loads the program (Ch.3), attaches it, generates
    // traffic with pktgen or a socket loop, and drains the ring in
    // batches (Ch.13.4). The method is printed so the number stays honest.
    let elapsed = start.elapsed();
    let per_packet_ns = elapsed.as_nanos() as f64 / PACKETS as f64;

    println!("kernel        : {}", kernel_release());
    println!("method        : userspace loop, ring drained in batches");
    println!("packets       : {PACKETS}");
    println!("total         : {elapsed:?}");
    println!(
        "per-packet    : {per_packet_ns:.1} ns  ({} Mpps)",
        (1e9 / per_packet_ns / 1e6) as u32
    );
}

The two disciplines in the demo are the ones the whole chapter stands on: the method is printed with the result (so nobody quotes the number without its conditions), and the number is expressed per packet (so the budget comparison of Section 13.1 works). When the capstone claims “the data path does X Mpps”, it will claim it with the kernel, the NIC, and the method attached.

13.6 The Bigger Picture: NUMA, IRQs, and the Node

The final piece of the performance model is the one Chapter 1 started: the receive path runs on the CPU the NIC’s RSS assigned, in softirq context. The consequences for eBPF performance are physical:

• Pin the CPUs. IRQ affinity, taskset for the userspace reader, and busy-polling (SO_BUSY_POLL) keep the NIC, the softirq, the ring reader, and the map’s per-CPU slots on the same NUMA node - the cache and memory-locality rules of the C++ book, applied to the data path.
• Count the per-CPU slots. A per-CPU map on a 64-core node has 64 slots; the sum-over-CPUs read (Chapter 4) crosses nodes on NUMA machines, and the reader should run on the node that owns the most traffic.
• Watch the softirq budget. If the poll loop of Chapter 1 is starved (the machine is busy elsewhere), the NIC queues fill, packets drop, and no eBPF optimisation fixes a CPU shortage.

None of this is eBPF-specific - it is the systems discipline of Part V applied to a program that happens to live in the kernel. The eBPF-specific lesson is the smaller one: the program is cheap; the placement is everything.

Hands-On Lab

# 1. The honest benchmark skeleton (records its own conditions).
cd code/ch13_performance/userspace && cargo run --release

# 2. The profiling loop on a loaded program.
sudo perf record -g -a -- sleep 10    # while traffic flows
sudo perf report | grep bpf_prog_     # the JIT symbol's share of cycles
sudo bpftool prog show name backend_lb   # runtime ns + packet counts

# 3. The map-vs-map experiment: PERCPU_ARRAY counter vs HASH counter at
#    1 Mpps; the contention is visible in `bpftool map show` and in perf.

The rule the lab practises: every number carries its kernel, NIC and method (rule 8), and the budget is accounted BEFORE the program is written - 4000/R cycles at R Mpps (Chapter 13.1).

Summary

• The methodology is accounting: budget = cores * freq / rate; count the lookups, checks, and helpers before writing the program.
• The cost table’s shape: maps are cheap, hooks decide the magnitude, helpers hide the variance - per-CPU arrays for hot paths, one composite key instead of two lookups, ring batching on the read side.
• The verifier’s instruction budget is a design constraint: a program that needs more instructions is a design that moved the wrong work into the kernel.
• Profile with perf + bpftool prog dump jited + map stats; the JIT output is small enough to read, the map stats are accurate enough to trust.
• Benchmarks are reproducible and honest: kernel, NIC, method, and per-packet cost printed together (rule 8).
• The node matters: pinning, NUMA, and softirq budget dominate any program-level optimisation - placement is everything.

Next: Chapter 14 is where the rejected programs get fixed - testing, debugging, and shipping eBPF in Rust, from verifier logs to CI.

Chapter 14: Testing, Debugging & Shipping eBPF in Rust

“Your eBPF program will be rejected. The question is whether the rejection is a compiler error or an incident. The difference is the discipline of this chapter: test the program before the kernel does, read the verifier like a compiler, and ship the diagnosis with the code.”

The hardest part of eBPF development is not writing programs; it is the moment the verifier disagrees with you - and the moments after, when a program that loaded silently does the wrong thing. This chapter is the complete debugging and testing playbook: reading verifier logs, using bpftool and bpftrace to inspect a running system, writing tests that catch rejections and logic bugs before they reach the kernel, testing the userspace control plane, and the CI setup that makes all of it run on every push. By the end, the failure modes of every earlier chapter - bounds violations, map contract mismatches, attach failures, silent drops - have a name and a recipe.

14.1 Reading the Verifier Like a Compiler

The verifier rejection of Chapter 2 deserves a full decode. The canonical log line:

12: (71) r1 = *(u8 *)(r2 + 20)     ; R2=pkt(off=14,r=14,imm=0)
invalid access to packet, off=20 size=1, R2 has only 14 bytes of readable data

The vocabulary: r2 is a packet pointer whose known-readable region is 14 bytes (the Ethernet header, r=14); the program tried to read byte 20. The verifier’s log is positional - it names the instruction, the register, and the property it could not prove. The debugging loop:

1. Read the register states, not just the message. R2=pkt(off=14,r=14) tells you what the verifier knows - usually “the bounds check you wrote covered 14 bytes, and then you read past it”.
2. Find the missing check. The fix is almost always a bounds comparison before the access: if data + 34 > data_end { return PASS }.
3. Check the obvious suspects first: helper calls that invalidate packet pointers (Chapter 5), structs without #[repr(C)], usize in a shared struct (Chapter 3’s contract rules).

The discipline that prevents most rejections: write the check before the access, every time, and review the pair together. The code review rule for eBPF code is “every read of packet memory must be preceded, within five lines, by the bounds check that justifies it.”

14.2 bpftool: The Runtime Truth

bpftool is the swiss army knife of the BPF subsystem, and the debugging session for any mystery is a sequence of its commands:

# What is attached, where, and how big is it?
bpftool prog list                        # all loaded programs
bpftool prog show id 123                 # one program's details
bpftool prog dump jited id 123           # the native code (Ch. 13)
bpftool prog dump xlated id 123          # the verified BPF instructions

# The maps: sizes, usage, and the live counters.
bpftool map list
bpftool map dump name ALLOW              # see the actual key/value entries

# The BTF type info that CO-RE relocations used (Ch. 2).
bpftool btf dump file /sys/kernel/btf/vmlinux | head

# Live tracing of every load/attach on the system.
bpftrace -e 'tracepoint:syscalls:sys_enter_bpf { printf("bpf() from %s\n", comm); }'

The key habit: when a program misbehaves, look at the maps first. A program that “does nothing” is usually a program whose map lookup misses - bpftool map dump shows you the keys that exist, and the miss becomes obvious (wrong key layout, wrong endianness, wrong composite key construction - the Chapter 4 contract bugs). The map is the program’s visible state, and it is fully inspectable while the program runs.

14.3 bpftrace: Confirm the Model Before You Debug It

bpftrace is the one-liner layer on top of eBPF, and it is the fastest way to confirm or refute a hypothesis about the kernel, not your program. Before debugging your tc classifier, check the premise with a one-liner:

# Hypothesis: retransmits spike when the backend pool shrinks.
bpftrace -e 'k:tcp_retransmit_skb { @[nsecs/1e6] = count(); }'

# Hypothesis: packets are dropped in the stack, not by my XDP program.
bpftrace -e 'tracepoint:skb:kfree_skb { @[args->reason] = count(); }'

The habit is the one the C++ book calls “verify the premise”: if your program is not doing what you expect, first establish what the kernel is actually doing - bpftrace answers that in one line, and the answer often redirects the whole investigation from your code to the model underneath.

14.4 Testing the Kernel Side: Rejection Tests and Logic Tests

The kernel side of an Aya project is hard to unit test directly (it needs the kernel), but the logic is testable in two ways:

Source: code/ch14_testing_debugging/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch14_testing_debugging/kernel/src/main.rs
// The kernel program: a thin shell over the host-testable parse module.
#![no_std]
#![no_main]

mod parse;

use aya_bpf::{macros::xdp, programs::XdpContext};
use aya_bpf::bindings::xdp_action::*;

#[xdp]
pub fn xdp_parse(ctx: XdpContext) -> u32 {
    let frame = unsafe { parse::frame(ctx.data(), ctx.data_end()) };

    // One pure function call: None means non-TCP or truncated - pass.
    match parse::dst_port(frame) {
        Some(port) if port == parse::SERVICE_PORT => XDP_DROP, // demo rule
        _ => XDP_PASS,
    }
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

The pure function itself lives in kernel/src/parse.rs - no panics, no BPF, no no_main attributes, just the bounds logic:

Source: code/ch14_testing_debugging/kernel/src/parse.rs

#![allow(unused)]
fn main() {
// code/ch14_testing_debugging/kernel/src/parse.rs
// Pure, total packet parsing: no unsafe, no kernel calls, no allocation.
// The program turns (data, data_end) into a slice ONCE, then all parsing
// is ordinary slice logic - bounds-checked, idiomatic, host-testable.
// The same module is exercised by `cargo test` on the host (Ch.14).
use core::mem::size_of;

// Build a slice over the DMA'd frame. The verifier accepts this because
// the length is derived from data_end - data (Ch.2's bounds discipline).
#[inline(always)]
pub unsafe fn frame<'a>(start: usize, end: usize) -> &'a [u8] {
    debug_assert!(end >= start, "data_end must not precede data");
    core::slice::from_raw_parts(start as *const u8, end - start)
}

#[inline(always)]
pub fn u8_at(f: &[u8], off: usize) -> Option<u8> {
    f.get(off).copied()
}

#[inline(always)]
pub fn u16_be(f: &[u8], off: usize) -> Option<u16> {
    let s = f.get(off..off + size_of::<u16>())?;
    Some(u16::from_be_bytes([s[0], s[1]]))
}

#[inline(always)]
pub fn u32_be(f: &[u8], off: usize) -> Option<u32> {
    let s = f.get(off..off + size_of::<u32>())?;
    Some(u32::from_be_bytes([s[0], s[1], s[2], s[3]]))
}

// ---- Chapter 14's testable core: the parse logic, pure and total --------

pub const ETH_IP: usize = 14 + 20;
pub const SERVICE_PORT: u16 = 8080;

/// Returns the dst port for an IPv4/TCP frame, None otherwise.
pub fn dst_port(frame: &[u8]) -> Option<u16> {
    if u16_be(frame, 12)? != 0x0800 || u8_at(frame, 14 + 9)? != 6 {
        return None;
    }
    u16_be(frame, ETH_IP + 2)
}
}

The testable design: the packet logic is a pure function over a slice, which is exactly the slice the program hands it. The same function can be compiled into the kernel program and into a native test binary (the #[cfg(test)] module in the same crate, or a tests/ integration test against the common crate). The rejection tests are then ordinary Rust:

Source: code/ch14_testing_debugging/kernel/tests/parse.rs

#![allow(unused)]
fn main() {
// code/ch14_testing_debugging/kernel/tests/parse.rs
// Host-run tests against the EXACT module the kernel program uses.
// No kernel, no BPF - the verifier's bounds discipline, checked by the
// Rust test harness first (Ch.14.4).
#[path = "../src/parse.rs"]
mod parse;

fn tcp_frame(dport: u16) -> Vec<u8> {
    let mut f = vec![0u8; parse::ETH_IP + 8];
    f[12..14].copy_from_slice(&0x0800u16.to_be_bytes()); // IPv4
    f[14 + 9] = 6;                                      // TCP
    f[parse::ETH_IP + 2..parse::ETH_IP + 4].copy_from_slice(&dport.to_be_bytes());
    f
}

#[test]
fn rejects_truncated_frames() {
    let short = tcp_frame(8080);
    assert_eq!(parse::dst_port(&short[..parse::ETH_IP + 1]), None);
}

#[test]
fn rejects_non_tcp() {
    let mut f = tcp_frame(8080);
    f[14 + 9] = 17; // UDP
    assert_eq!(parse::dst_port(&f), None);
}

#[test]
fn finds_tcp_port() {
    assert_eq!(parse::dst_port(&tcp_frame(8080)), Some(8080));
}
}

The pattern is the one every serious eBPF project uses: host-run tests for the parse logic, kernel-run tests for the load/attach, and the same function in both places. The rejection that would have taken an hour to read from a verifier log becomes a test failure that names the bug.

14.5 Testing the Userspace Side: The Control Plane Is a Program

The userspace side has no excuse - it is ordinary Rust, and it gets ordinary tests: the map contract (insert what the kernel expects, read what it wrote), the ring draining (batch reads, handle overflow), and the attach errors (wrong interface, missing privileges, kernel config). The tests that matter most are the contract tests: a common struct that crosses the boundary must produce exactly the byte layout both sides expect, and a #[test] that checks size_of and offsets catches the #[repr(C)] mistakes of Chapter 3 before they become map garbage.

Source: code/ch14_testing_debugging/userspace/tests/contract.rs

#![allow(unused)]
fn main() {
// code/ch14_testing_debugging/userspace/tests/contract.rs
// The contract test: the shared struct's layout is the agreement between
// kernel and userspace. If this fails, every map read is garbage (Ch.14.5).
#[test]
fn contract_layout_is_stable() {
    use core::mem::{align_of, size_of};

    // The ring event of Chapter 9: fixed size, C layout, no surprises.
    #[repr(C)]
    struct DenyEvent {
        dst_ip: u32,
        proto: u8,
        ts: u64,
    }
    assert_eq!(size_of::<DenyEvent>(), 16);
    assert_eq!(align_of::<DenyEvent>(), 8);
}
}

14.6 Shipping: CI, Privileges, and the Load Test

The CI setup for an eBPF repo is the point where the discipline becomes a pipeline:

1. Lint and build everything: cargo fmt --check, cargo clippy on the userspace and common crates, cargo build --target bpfel-unknown-none on the kernel crate - every push.
2. Run the host tests: the parse tests and contract tests of Sections 14.4-14.5 - no kernel needed, fast, and they catch 80% of bugs.
3. Run the kernel tests on a real kernel: a CI job on an ubuntu-latest runner (which has BTF enabled by default on modern versions) that loads the programs against a veth pair, attaches them, generates traffic, and asserts the events arrive. This is the test that the local machine (macOS, Windows, or a locked-down Linux) cannot run - which is why the book’s CONTRIBUTING.md asks contributors to state their platform, and why the workflow runs the kernel job regardless.
4. The load test as a gate: the benchmark of Chapter 13, run in CI, failing on a regression beyond a threshold - the honest, repeatable performance number, enforced.

The final shipping concern is privileges: loading eBPF needs CAP_BPF (+ CAP_PERFMON for tracing, CAP_SYS_ADMIN on older kernels), which is why production deployments run the loader in a privileged init container and the runtime (after attach) needs nothing - the maps are updated by the control plane, and the data path runs unattended. That separation is the security posture of Chapter 15, made operational.

Hands-On Lab

# 1. The tests that need no kernel: run them anywhere.
cd code/ch14_testing_debugging/kernel
cargo test -- --nocapture        # parse tests against the pure module

# 2. Break a bound on purpose and watch the test catch it.
#    (change `ETH_IP + 2` to `ETH_IP + 4` in parse.rs, re-run: the
#     truncation test fails - that is a verifier rejection, caught first)

# 3. The runtime truth, when the program is loaded.
sudo bpftool prog list
sudo bpftool map dump name BACKENDS
sudo bpftrace -e 'k:tcp_retransmit_skb { @ = count(); }'

The CI ladder from Section 14.6 in practice: fmt/clippy, host tests, kernel load tests on a real runner, benchmark gate - every push.

Summary

• Read the verifier log as a compiler error: the register states name the missing proof; the fix is the bounds check before the access.
• bpftool is the runtime truth: program list, JIT dump, and - most often - the map dump that shows the contract bug.
• bpftrace confirms the premise: establish what the kernel is doing before debugging what your program is not.
• The testable design: packet logic as a pure slice function, tested on the host; the same function compiled into the kernel program.
• The userspace side gets ordinary tests: the layout contract test catches the #[repr(C)] bugs before they reach a map.
• CI runs the ladder: fmt/clippy/build, host tests, kernel load tests, benchmark gate - and production separates privileged loader from unprivileged runtime.

Next: Chapter 15 closes Part V with the security model itself - what eBPF can do, what it cannot, and where the attack surface really is.

Chapter 15: Security - The eBPF Attack Surface

“eBPF is the safest way to run code in the kernel, and the most dangerous thing you can attach to the kernel if you are wrong about who controls it. Both sentences are true, and the security model is the difference between them.”

Every chapter has assumed a trusted operator loading trusted programs. This chapter examines what that trust means: the capabilities and lockdown mechanisms that gate bpf(), the unprivileged path and its history, what an attacker with a loaded program can and cannot do, and how eBPF is itself the defense - the runtime-security agents (Falco, Tetragon, the Cilium security stack) that watch the kernel from inside. The eBPF security model is the verifier of Chapter 2 plus the capability system of the kernel, and this chapter is where you learn to reason about both at once.

15.1 The Privilege Model: Capabilities, Not Root

Loading eBPF programs and creating maps requires capabilities - the kernel’s fine-grained privilege tokens - not blanket root. The relevant set, on modern kernels:

The consequence for deployment is the one Chapter 14 stated: the loader needs the capabilities; the runtime (after attach) does not. Production systems run the loader in a privileged init container, drop capabilities after load where possible, and never give the data path’s consumers more than they need. This is the same least-privilege discipline as any systems service - eBPF just makes the boundary explicit and inspectable.

15.2 Unprivileged BPF: The Attack Surface That Was

Older kernels allowed unprivileged users to load a restricted class of BPF programs (socket filters, and later a few more) - the legacy of the 1992 Berkeley Packet Filter, which was designed for exactly that. The history since is a list of CVEs: Spectre-class side channels through the verifier’s value tracking (CVE-2017-16995 was the famous one), JIT bugs, map accounting bugs. The kernel’s response was layered:

1. kernel.unprivileged_bpf_disabled (and sysctl hardening): the unprivileged path is off by default on virtually all distributions.
2. The verifier hardened repeatedly: speculation barriers, precise value tracking, the “speculative execution” mitigations that made the Spectre class expensive to exploit.
3. Lockdown mode (lockdown=confidentiality): bpf() is gated entirely when the kernel is in lockdown, closing the last paths.

The lesson for the systems engineer is not the CVE list; it is the default: assume unprivileged BPF is unavailable, design the loader to need capabilities, and treat “can this user load BPF?” as a security question on the same level as “can this user load a kernel module?” - the answer is checked by the same trust boundary.

15.3 What a Loaded Program Can Do: The Trust Model, Precisely

A loaded eBPF program is kernel code - but kernel code constrained by the verifier. The precise trust model:

• Can: read and write its maps (bounded), call the approved helpers, read the packet/skb it was given, inspect kernel memory only through BTF-typed read access in tracing programs (bounded by the verifier’s provenance checks).
• Cannot: call arbitrary kernel functions, dereference arbitrary pointers, read/write kernel memory outside its maps and context, loop unboundedly, or escape into kernel control flow.

The honest caveat: “cannot” is “the verifier proved it cannot, as of this kernel version”. The verifier is the attack surface - which is why its bugs are kernel security bugs, why the subsystem is one of the most audited in the kernel, and why your own programs should be written to reduce the verifier’s work (simple shapes, no pointer games). A program that needs an exotic verifier feature is a program that should be rewritten - for security reasons, not just style.

The second half of the trust model is what the program’s operator can do, and that is the deployment question: a program attached to the right hooks with the right maps can see every packet, every socket operation, every syscall of every process. Runtime-security agents exist precisely to deploy that power as defense.

15.4 eBPF as Defense: The Runtime Security Stack

The flip side of the attack surface is the defensive capability: eBPF programs attached to syscall tracepoints and cgroup hooks can watch everything without modifying the target processes - no ptrace, no injection, no performance cliff. This is the architecture of the modern runtime security agents:

• Falco (the pioneer): kprobes/tracepoints on syscalls and container lifecycle events, rules evaluated in userspace, alerts on suspicious behaviour - the observability pattern of Chapter 7, productised as detection.
• Tetragon (Cilium’s security agent): policy enforcement in the kernel - the policy map of Chapter 12 applied to process execution and file access, with the enforcement happening in the BPF program itself, not after a userspace round trip.
• The Cilium security stack: the network policies of Chapter 12 (identity-based), plus the Hubble flow log as the audit trail.

The pattern is the book’s pattern one more time: the kernel produces events; the control plane decides; the data path enforces. The difference is the subject: instead of packets, the subject is processes and syscalls - the cgroup hooks of Chapter 9 and the tracepoints of Chapter 7, applied to execve, open, connect, write.

Source: code/ch15_security/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch15_security/kernel/src/main.rs
// A minimal runtime-security probe: stream execve events to the ring.
// The Falco/Tetragon shape in miniature - kernel produces, userspace
// decides (Ch.15.4).
#![no_std]
#![no_main]

use aya_bpf::{
    macros::{map, tracepoint},
    maps::RingBuf,
    programs::TracePointContext,
};

const PID_OFF: usize = 0;   // sched/sched_process_exec argument block
const COMM_OFF: usize = 16;

#[repr(C)]
pub struct ExecEvent {
    pub pid: u32,       // pid of the exec'ing process
    pub uid: u32,       // user id (filled by the real agent from the task)
    pub comm: [u8; 16], // process name
}
#[map]
static EXECS: RingBuf = RingBuf::with_max_entries(8192, 0);

#[tracepoint]
pub fn sched_process_exec(ctx: TracePointContext) -> i32 {
    let Ok(ev) = try_exec(&ctx) else {
        return 0; // truncated tracepoint block: skip
    };
    let Ok(entry) = EXECS.reserve::<ExecEvent>(0) else {
        return 0; // ring full: drop the event
    };
    unsafe { entry.write(ev) };
    entry.submit(0);
    0
}

fn try_exec(ctx: &TracePointContext) -> Result<ExecEvent, i32> {
    let pid: u32 = unsafe { ctx.read_at(PID_OFF)? };
    let comm: [u8; 16] = unsafe { ctx.read_at(COMM_OFF)? };
    Ok(ExecEvent { pid, uid: 0, comm })
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

The security property that makes this defense rather than surveillance: the program is read-only, bounded, and attached by a trusted agent whose policy decides what the stream means. The same program shape in the wrong hands is espionage; in the right hands it is the runtime firewall of the next decade. The difference is the trust boundary - capabilities, policy, and audit - which is exactly what Sections 15.1-15.3 described.

15.5 The Audit Trail: eBPF Programs as Evidence

One more property completes the security picture: eBPF is inspectable. bpftool prog list, the pinned programs under /sys/fs/bpf/, and the BTF info mean that a machine’s loaded programs are enumerable - the forensic question “what eBPF is running on this host?” has an answer, and it is a bpftool one-liner. Security systems built on eBPF should therefore ship their own identity: a pinned program, a version in a map, an audit log in the ring. When the capstone (Chapter 16) deploys its programs, it will pin them under /sys/fs/bpf - making them visible, named, and accountable, the way production eBPF should be.

Hands-On Lab

# 1. Load the exec probe and stream events.
cd code/ch15_security/kernel && cargo build --release --target bpfel-unknown-none
sudo bpftool prog loadall <elf> /sys/fs/bpf/sec 2>&1 || echo "need CAP_BPF/CAP_PERFMON"
# (or attach through the userspace loader of Chapter 3)

# 2. The privilege story, made concrete.
cat /proc/sys/kernel/unprivileged_bpf_disabled   # 2 = locked down
capsh --print | grep cap_bpf                     # what the loader holds

# 3. The audit story: eBPF is inspectable by design.
sudo bpftool prog list | head
ls /sys/fs/bpf/                 # pinned programs ship their identity

The security model is two halves: the verifier proves what a program CANNOT do (Ch.2), and capabilities decide WHO loads it (Ch.15.1). Both halves are checkable on a live host with the commands above.

Summary

• eBPF loading is gated by capabilities, not root: CAP_BPF, CAP_PERFMON, CAP_NET_ADMIN; the loader needs them, the runtime does not.
• Unprivileged BPF is the historical attack surface: disabled by default, hardened verifier, lockdown mode - assume it is unavailable.
• A loaded program is kernel code constrained by the verifier: maps and helpers yes, arbitrary memory no - and the verifier itself is the attack surface, which is why simple program shapes are a security property.
• eBPF as defense: Falco and Tetragon watch syscalls and enforce policy in the kernel - the book’s pattern (kernel produces, userspace decides, data path enforces) applied to processes.
• eBPF is inspectable by design: bpftool enumerates loaded programs; production deployments pin and name them for audit.

Next: Part VI. Chapter 16 assembles everything - the capstone service-mesh data path in Rust, running against a real cluster.

Chapter 16: Capstone - A Service-Mesh Data Path in Rust with eBPF

“Every chapter taught you a program. This chapter teaches you a system: the pieces you built - the XDP load balancer, the sockmap redirect, the policy map, the flow ring - wired into one Rust workspace, deployed to a real cluster, and measured. This is the book, assembled.”

The capstone is the book’s promise made concrete: a service-mesh data path in Rust with eBPF - the same architecture as the Cilium stack of Chapters 11-12, built from the primitives of Chapters 3-9, running against a kind cluster. It is deliberately minimal so that every piece is legible, and deliberately complete so that every piece is real: an XDP program that load-balances traffic by a hash, sockmap programs that redirect connections between replicas in the kernel, a policy map that decides what may pass, and a ring buffer that ships flow events to a Hubble-style observer. By the end you will have written - and run - a one-node Cilium.

16.1 The Architecture: The Book, Assembled

The capstone is one workspace, three kernel programs, and one userspace control plane:

code/ch16_capstone/
├── Cargo.toml                 # workspace
├── common/                    # shared #[repr(C)] contract (Ch. 3)
│   └── src/lib.rs             #   FlowEvent, Backend, PolicyRule
├── kernel/                    # the three data-path programs
│   ├── src/lb.rs              #   XDP: hash -> backend (Ch. 5)
│   ├── src/mesh.rs            #   sockmap SK_MSG redirect (Ch. 8)
│   ├── src/flow.rs            #   tracepoint -> flow ring (Ch. 7, 4)
│   └── src/main.rs
└── userspace/                 # the control plane
    ├── src/main.rs            #   load, attach, fill maps, drain rings
    └── src/observer.rs        #   the Hubble-style flow reader

The mapping to the book is the point of the diagram:

16.2 The Kernel Side: Three Programs, One Contract

The common crate is the contract the three programs and the control plane share - the Chapter 3 discipline in action:

Source: code/ch16_capstone/common/src/lib.rs

#![allow(unused)]
fn main() {
// code/ch16_capstone/common/src/lib.rs
// The shared contract: every type that crosses the kernel/userspace
// boundary lives here, #[repr(C)], fixed-size, one source of truth (Ch.3).
#![no_std]

// One flow event, shipped through the ring (Ch.4).
#[repr(C)]
pub struct FlowEvent {
    pub src_ip: u32,      // network byte order, as parsed from the packet
    pub dst_ip: u32,
    pub src_port: u16,
    pub dst_port: u16,
    pub verdict: u8,      // 1 = allowed, 0 = denied (fail-open default)
    pub ts: u64,          // bpf_ktime_get_ns
}

// The load-balancer table entry (Ch.5).
#[repr(C)]
pub struct Backend {
    pub ifindex: u32,     // redirect target interface
    pub flags: u32,       // reserved
}

// The policy entry (Ch.4, 12): key = dst ip, value = allow(1)/deny(0).
pub type PolicyRule = u8;

// Compile-time layout check - the contract test of Ch.14, run at build.
pub const _: () = {
    assert!(core::mem::size_of::<FlowEvent>() == 24);
    assert!(core::mem::align_of::<FlowEvent>() == 8);
};
}

The load balancer is the Chapter 5 program with a hash: instead of looking up a fixed port, it hashes the flow tuple and picks a backend from a small array - the Katran pattern of Chapter 5.6:

Source: code/ch16_capstone/kernel/src/lb.rs

#![allow(unused)]
fn main() {
// code/ch16_capstone/kernel/src/lb.rs
// The XDP load balancer: hash the flow tuple, pick a backend, redirect.
#![no_std]

use aya_bpf::{
    macros::{map, xdp},
    maps::Array,
    programs::XdpContext,
};
use aya_bpf::bindings::xdp_action::*;

const MAX_BACKENDS: u32 = 8;
const ETH_IP: usize = 14 + 20;
const HASH_SEED: u32 = 0x9e37_79b9; // a large odd constant (Knuth's 2^32/golden ratio)

#[repr(C)]
pub struct Backend {
    pub ifindex: u32,
    pub flags: u32,
}
#[map]
static BACKENDS: Array<Backend> = Array::with_max_entries(MAX_BACKENDS, 0);

// Multiply-xor hash of the 4-tuple: enough spread for backend selection,
// no loops - the verifier needs no proof of termination (Ch.16.2).
fn flow_hash(src: u32, dst: u32, sport: u16, dport: u16) -> u32 {
    let mut h = src ^ dst;
    h = h.wrapping_mul(HASH_SEED);
    h ^= (u32::from(sport) << 16) | u32::from(dport);
    h.wrapping_mul(HASH_SEED)
}

// The 4-tuple, parsed with the bounds-checked slice idiom of Ch.5.
fn tuple(frame: &[u8]) -> Option<(u32, u32, u16, u16)> {
    // ethertype at offset 12: only IPv4 (0x0800) has a fixed TCP offset.
    if frame.get(12..14)? != b"" {
        return None;
    }
    let s = frame.get(14 + 12..14 + 16)?;
    let d = frame.get(14 + 16..14 + 20)?;
    let sp = frame.get(ETH_IP..ETH_IP + 2)?;
    let dp = frame.get(ETH_IP + 2..ETH_IP + 4)?;
    Some((
        u32::from_be_bytes([s[0], s[1], s[2], s[3]]),
        u32::from_be_bytes([d[0], d[1], d[2], d[3]]),
        u16::from_be_bytes([sp[0], sp[1]]),
        u16::from_be_bytes([dp[0], dp[1]]),
    ))
}

#[xdp]
pub fn lb(ctx: XdpContext) -> u32 {
    let frame = unsafe {
        core::slice::from_raw_parts(
            ctx.data() as *const u8,
            ctx.data_end() - ctx.data(),
        )
    };

    let Some((src, dst, sport, dport)) = tuple(frame) else {
        return XDP_PASS; // not IPv4 or truncated: fail open
    };

    // Array lookup cannot fail (Ch.4); an empty slot passes.
    let idx = (flow_hash(src, dst, sport, dport) % MAX_BACKENDS) as usize;
    let backend = unsafe { BACKENDS.get_ptr(idx) };
    match unsafe { backend.as_ref() } {
        Some(be) if be.ifindex != 0 => aya_bpf::helpers::bpf_redirect(be.ifindex, 0),
        _ => XDP_PASS,
    }
}
}

The sockmap program is Chapter 8’s SK_MSG verdict verbatim (the mesh half), and the flow program is Chapter 7’s tracepoint plus Chapter 4’s ring. Together they are the three faces of the book’s data path: the packet face (XDP), the socket face (sockmap), and the observability face (tracepoint + ring) - all three sharing the one common contract.

16.3 The Control Plane: Load, Attach, Fill, Drain

The userspace side is Chapter 3’s lifecycle, run for three programs, with the control-plane duties of Chapters 4-5:

Source: code/ch16_capstone/userspace/src/main.rs

// code/ch16_capstone/userspace/src/main.rs
// The control plane: load, attach all three programs, fill maps, drain
// the flow ring. Chapter 3's lifecycle, run three times (Ch.16.3).
mod observer;

use aya::{
    maps::{Array, RingBuf, SockHash},
    programs::{SockMap, TracePoint, Xdp},
    Ebpf,
};
use common::{Backend, FlowEvent};
use std::error::Error;
use std::time::Duration;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const DEFAULT_IFACE: &str = "eth0";
const BACKEND_IFINDEX: u32 = 2;
const POLL_INTERVAL: Duration = Duration::from_millis(100);

fn main() -> Result<(), Box<dyn Error>> {
    let iface = std::env::args().nth(1).unwrap_or_else(|| DEFAULT_IFACE.to_string());
    let mut ebpf = Ebpf::load(KERNEL_ELF)?;

    // --- Chapter 5: the XDP load balancer ---
    let lb: &mut Xdp = ebpf.program_mut("lb")?.try_into()?;
    lb.load()?;
    lb.attach(&iface, aya::programs::XdpFlags::default())?;

    // --- Chapter 8: the sockmap mesh ---
    let mesh: &mut SockMap = ebpf.program_mut("msg_redirect")?.try_into()?;
    mesh.load()?;
    mesh.attach(&aya::programs::SockMapAttachType::MsgVerdict)?;

    // --- Chapter 7: the flow tracepoint ---
    let flow: &mut TracePoint = ebpf.program_mut("sched_process_exec")?.try_into()?;
    flow.load()?;
    flow.attach("sched", "sched_process_exec")?;

    // --- Chapter 4: fill the maps (the control-plane half) ---
    let mut backends: Array<_, Backend> = ebpf.map_mut("BACKENDS")?.try_into()?;
    backends.set(0, Backend { ifindex: BACKEND_IFINDEX, flags: 0 }, 0)?;
    println!("lb: backend[0] = ifindex {BACKEND_IFINDEX}");

    let _conns: SockHash<_, u32> = ebpf.map_mut("CONNS")?.try_into()?;
    // (In production the mesh accepts a connection, dials the backend,
    // and inserts both sockets - Chapter 8's control-plane pattern.)

    // --- Chapter 4/12: drain the flow ring like a one-node Hubble ---
    let mut flows: RingBuf = ebpf.map_mut("FLOWS")?.try_into()?;
    println!("capstone running: lb + mesh + flow observer on {iface}; Ctrl-C to exit");

    loop {
        std::thread::sleep(POLL_INTERVAL);
        for item in flows.iterator() {
            let ev: FlowEvent = item.read()?;
            observer::log(&ev);
        }
    }
}

Every line maps to a chapter; that is the capstone’s design goal. The control plane is deliberately unoptimised (a 100 ms poll) because the data path is the point - and the separation between the two is the book’s central discipline, now visible in one file.

16.4 Running It: kind, Cilium, and the Two Viewpoints

The capstone runs against the cluster you built in Chapter 11.5. The procedure, with the verification each step demands:

# 1. Build the kernel side and the userspace side.
cd code/ch16_capstone/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && cargo build --release

# 2. The cluster: kind + Cilium (from Chapter 11), so the mesh has
#    pods and policies to route between.
kind create cluster --name ebpf-book
helm install cilium cilium/cilium --namespace kube-system \
  --set kubeProxyReplacement=true

# 3. Run the capstone on the node. (In production this is a daemonset
#    with the loader's capabilities - Chapter 15.)
sudo ./target/release/userspace eth0

# 4. Generate traffic between two pods; watch BOTH viewpoints.
kubectl run client --image=nicolaka/netshoot -- sleep 3600
kubectl exec client -- curl -s http://api:443/healthz

# Viewpoint A - the kernel (Chapter 14): the XDP program, live.
bpftool prog list | grep xdp
bpftool map dump name BACKENDS

# Viewpoint B - the ring: the flow observer's stream, plus Hubble's.
# (the capstone's stdout shows FlowEvent lines; hubble shows the same
#  flows with pod names - Chapter 12's enrichment.)
hubble observe --from-pod client

The verification loop is the book’s loop: the same traffic visible from the kernel (bpftool) and from the ring (the observer), with Hubble as the reference implementation. When the three agree, the capstone is correct; when they disagree, you have a debugging session straight out of Chapter 14.

16.5 Measuring It: The Chapter 13 Budget, Applied

The capstone ships with the benchmark of Chapter 13: generate load with pktgen (the kernel’s packet generator) or a curl loop, and measure the data path from both sides - perf on the XDP program, the ring’s event rate as the observer’s throughput, and the bpftool prog show run-time counters. The honest claims you will be able to make, each with its method attached:

• The XDP LB runs at Mpps scale on a single core (a hash, an array lookup, a redirect - the Chapter 13 cost table says so, the benchmark proves it).
• The sockmap mesh moves connections without userspace round trips - the proxy-vs-sockmap latency comparison of Chapter 8, measured with the same load.
• The flow ring ships events at the tracepoint rate with bounded CPU - the observability cost of Chapter 13’s table, amortised by batching.

16.6 Where the Capstone Goes From Here

The capstone is complete, not finished - the epilogue’s list, applied:

1. Make the LB consistent-hashing (Maglev-style): replace the modulo with a backend-selection map so removing a backend does not reshuffle every flow.
2. Add the cgroup hook (Chapter 9): attach the policy program to the client pod’s cgroup so the mesh’s allow/deny decisions are per-pod.
3. Wire the observer to Hubble: replace the 100 ms poll with Cilium’s ring-reading pattern and push enriched flows to the Hubble API
   • the capstone becomes a plugin, not a demo.
4. Ship it: the daemonset, the capability model of Chapter 15, the CI of Chapter 14 (load test on a real kernel), the benchmark gate of Chapter 13.

Then, the book’s final instruction - the one every chapter was practicing: “Let me show you the code and the cluster.”

Hands-On Lab

# 1. Build everything.
cd code/ch16_capstone/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && cargo build --release

# 2. The cluster (from Chapter 11).
kind create cluster --name ebpf-book
helm install cilium cilium/cilium --namespace kube-system \
  --set kubeProxyReplacement=true

# 3. Run the capstone and generate traffic between two pods.
sudo ./target/release/userspace eth0 &
kubectl run client --image=nicolaka/netshoot -- sleep 3600
kubectl exec client -- curl -s http://api:443/healthz

# 4. The two viewpoints, one truth.
sudo bpftool prog list | grep -E 'xdp|sock|tracepoint'   # kernel side
# (the userspace stdout shows FlowEvent lines; hubble shows the same
#  flows enriched with pod names)
hubble observe --from-pod client

When the kernel viewpoint, the ring, and Hubble agree, the capstone is correct - and you have built a one-node Cilium (Chapters 11-12, assembled from Chapters 3-9).

Summary

• The capstone is one workspace, three kernel programs, one control plane: XDP LB (Ch. 5), sockmap mesh (Ch. 8), flow tracepoint (Ch. 7), all sharing one #[repr(C)] contract (Ch. 3).
• The common crate is the contract: one source of truth for the structs both sides read, with the compile-time layout check of Ch. 14.
• The control plane is Chapter 3’s lifecycle run three times: load, attach, fill maps, drain rings - the book’s two-plane discipline in one file.
• Run it against kind + Cilium, verify from two viewpoints - bpftool (the kernel) and the flow ring (userspace) - with Hubble as the reference.
• Measure with Chapter 13’s method: kernel, NIC, method, per-packet cost printed with every claim.
• The extensions are the book’s remaining chapters: consistent hashing, cgroup policy, Hubble integration, and the production ship of Ch. 14-15.

Epilogue - The Road Ahead

The book opened with a promise: that the kernel’s packet path is learnable from first principles, that eBPF makes it programmable, and that Rust makes that programming safe enough to trust. Sixteen chapters later, the promise has a shape. You have built the packet path, the eBPF virtual machine and its verifier, the map contract, the XDP and tc data planes, the socket layer, the Kubernetes service model, and a complete service-mesh data path - each one explained, each one measured, each one built from primitives you can now define from memory.

What You Now Know (and Can Derive)

The real inventory is not the topics; it is the derivations. You can now answer, from models rather than memory:

• Why a packet costs what it costs, and where eBPF hooks can cut the cost (Chapters 1, 7).
• Why the verifier rejects a program, and how to shape a program so it passes (Chapters 2, 14).
• Why a map type is right for an access pattern, and how the kernel and userspace sides of that contract stay in sync (Chapter 4).
• Why XDP runs before the stack and tc after, and what each placement buys you (Chapters 5-6).
• Why a sockmap redirect avoids a round trip through userspace, and how the socket layer makes it correct (Chapter 8).
• Why kube-proxy was the bottleneck, and how Cilium’s eBPF data plane replaced it (Chapters 10-11).
• Why eBPF is both the safest way to extend the kernel and a powerful attack surface, and how the kernel fences both sides (Chapter 15).

The book’s thesis - stated in the foreword and proven by the capstone - is that the kernel is not a black box; it is the next platform, and you now speak its language.

Where the Discipline Goes From Here

Every chapter ends with the same implicit instruction: apply this to something real. The natural next steps, in order of leverage:

1. Run the capstone, then break it. The Chapter 16 service-mesh data path is deliberately minimal. Add an L7 filter, a second backend, a latency histogram - and watch the verifier and the ring buffer tell you exactly where your design is too clever (Chapters 4-5, 14).
2. Trace something you run every day. Pick a process you care about and write an Aya program that counts its system calls, its TCP retransmits, or its page faults. The tracing skills of Chapters 7 and 9 are the fastest way to make eBPF second nature.
3. Read the kernel source. The hooks you used are a few hundred lines in net/core/dev.c, net/socket.c, and kernel/bpf/*.c. The model from Chapters 1-2 makes those files readable; the files will sharpen the model.
4. Contribute to Aya or Cilium. Both projects are welcoming, both are written (increasingly) in Rust, and both reward exactly the skills this book practices: verifier-friendly programs, careful map design, and userspace control planes with real error handling.
5. Teach it. The strongest test of a model is explaining it to someone else - on a whiteboard, to a mentee, or in the interview itself.

The Data Path, One Last Time

You are in the room. The interviewer asks, “Design a service-mesh data path for a Kubernetes cluster.” You draw the diagram from Chapter 16: an XDP program hashing packets to backends, sockmap programs redirecting sockets in the kernel, a ring buffer shipping events to a Hubble-style observer, and a Rust userspace control plane that programs all of it. You name the hook points and the map types. You say why there are no copies and no userspace round trips on the hot path. You volunteer the tradeoffs: XDP cannot parse L7, sockmap does not help UDP as much as TCP, the verifier caps your program size, and the control plane must be exactly as careful as any distributed system.

Then you say the sentence this book has been practicing: “Let me show you the code and the cluster.”

Good luck - and go move some packets.

Appendix A - Aya & eBPF API Reference

A compact reference for the names that recur through the book. The Aya version referenced throughout is the 0.13 line (aya and aya-bpf crates at 0.13.x); where the API differs in newer versions, the book’s comments say so. Check the crate docs for the exact signatures of your pinned version.

Userspace (aya crate)

Kernel side (aya-bpf crate)

Kernel constants worth remembering

The three syscalls that matter

Everything eBPF goes through bpf(2), and everything you attach to goes through the usual suspects:

All of them are wrapped by Aya; you will only meet the raw syscalls when debugging with strace or reading bpftool output (Chapter 14).

Appendix B - Networking & Kernel Notation

Every number and unit used in the book, defined once, with the reasoning behind each one. These are the figures you will quote when someone asks “how fast is this really?” - and the units that keep you honest when you answer.

Rates, sizes and units

The per-packet budget

The single most useful number in the book. For a machine with C cores running at F GHz, the time available per packet at R Mpps is:

cycle budget per packet = (C * F * 10^9) / (R * 10^6)   cycles/packet

Examples at 4 GHz total (e.g. a 4-core slice):

Latency ladder (typical Linux numbers)

Kernel machinery you will see in logs

Time notation

• ktime_get_ns - kernel monotonic clock in nanoseconds; the clock BPF programs use (Chapter 13).
• CLOCK_MONOTONIC - the userspace twin; immune to wall-clock jumps.
• p50 / p99 / p999 - percentile latency; network systems are judged on p99+ because tail latency is what users feel.

Use this appendix as your cheat sheet while reading; every claim in the book is stated so it can be checked against one of these rows.

Appendix C - Recommended Reading & Tools

The book is a living document and so is this appendix. Everything listed here either appears in the chapters or fills the gaps the chapters leave open on purpose.

The kernel, first hand

• The Linux kernel source, net/ and kernel/bpf/ in particular. Read net/core/dev.c (the packet path of Chapter 1), net/ipv4/tcp_input.c (the receive path of Chapter 7), net/core/sock_map.c (Chapter 8) and kernel/bpf/verifier.c (Chapter 2 - take it slowly). Nothing replaces it.
• BPF and XDP Reference Guide (Cilium docs). The canonical hook-by-hook reference; Chapters 5-6 and 10-12 are its practical shadow.
• kernel.org docs: Documentation/bpf/ for the instruction set, verifier, and CO-RE/BTF.

Books and papers

• Linux Kernel Networking: Implementation and Theory (Rosen) - the protocol internals behind Chapter 7.
• BPF Performance Tools (Gregg) - the tracing playbook behind Chapters 7, 13 and 15, with the perf numbers to steal.
• Systems Performance (Gregg) - the methodology: USE method, off-CPU analysis, and the latency ladder of Appendix B.
• TCP/IP Illustrated, Volume 1 (Stevens) - the protocol model; still the clearest statement of how TCP actually behaves.
• The Cilium documentation and the Cilium: BPF and XDP Reference Guide
  • the production data path of Chapters 10-12, with the real programs (written in C, read them alongside the Rust of this book).

Tools you should install

Where to follow the ecosystem

• Aya project (aya-rs) - Rust eBPF; docs, examples, and the aya-examples repository that the code companions of this book build on.
• Cilium - the reference deployment; watch its design documents for the datapath decisions explained in Chapters 10-12.
• bpf-next mailing list and the BPF kernel docs - where the hooks you use are designed; the bpf subsystem moves fast and the list is where the future of Chapters 13-15 is being written.

A final note on versions: eBPF moves quickly. When a number in this book disagrees with your kernel, trust the kernel - and tell us, so the book can catch up.