Chapter 6: Traffic Control & the Socket Filter Hooks

“XDP sees the packet before the kernel believes in it. tc sees the packet after the kernel has invested in it - and that investment buys you the whole protocol stack on both sides of the filter.”

Chapter 5 took the hook at the front door. This chapter takes the hooks between the stack’s halves: the traffic control (tc) layer, which runs egress after the stack has produced an skb and ingress before the stack consumes it, plus the classic socket filter hook where the Berkeley Packet Filter was born. Where XDP is a pre-protocol decision point, tc is a post-protocol inspection point: it sees skbs with routing, conntrack and socket context - and it can modify, redirect, drop, or rate them. This is the hook Cilium uses for most of its data path, and the hook where the trade-offs between “as early as possible” and “as informed as possible” become concrete.

6.1 The Two Worlds of Traffic Control

The tc subsystem predates eBPF by decades: it is the kernel’s queueing and shaping machinery (tc qdisc, tc filter). Its eBPF integration gave the ancient clsact class a modern superpower: an eBPF program attached as a classifier can decide a packet’s fate and return an action the kernel obeys. The modern attach point is the clsact qdisc, which provides two hook points per interface:

• ingress: runs on receive, after GRO and the early driver path, on the way to protocol dispatch (Chapter 1’s netif_receive_skb).
• egress: runs on transmit, after the stack has produced the skb (routing, socket, TCP segmentation decisions all made), before the driver’s transmit path.

Each hook runs an ordered list of filters; a clsact eBPF classifier returns TC_ACT_OK (accept), TC_ACT_SHOT (drop), TC_ACT_REDIRECT (redirect), or a few rarer actions, and may edit the skb in place.

6.2 XDP vs tc: The Decision Table

Every networking eBPF design eventually asks: XDP or tc? The answer is a table, not a preference:

The rule of thumb that the rest of the book uses: drop as early as you can justify; decide as late as you must. If the decision is “is this address on the ban list?” - XDP. If the decision is “does this connection belong to this pod’s policy?” - tc, because the socket and conntrack state that makes the answer correct only exists there.

6.3 The tc Program in Aya

Source: code/ch06_tc_hooks/kernel/src/main.rs

#![allow(unused)]
fn main() {
// code/ch06_tc_hooks/kernel/src/main.rs
// A tc ingress classifier: one composite-key lookup against an allowlist.
#![no_std]
#![no_main]

mod parse;

use aya_bpf::{
    macros::{classifier, map},
    maps::HashMap,
    programs::SchedClassifierContext,
};
use aya_bpf::bindings::tc_actions::*;

const MAX_RULES: u32 = 1024;
const ACTION_ALLOW: u32 = 1;

#[repr(C)]
pub struct Rule {
    pub action: u32, // 1 = allow, 0 = drop
}
#[map]
static ALLOW: HashMap<u64, Rule> = HashMap::with_max_entries(MAX_RULES, 0);

#[classifier]
pub fn ingress_check(ctx: SchedClassifierContext) -> i32 {
    // tc contexts hand the frame as an Option; a missing skb is not ours
    // to judge - the stack sorts it out.
    let Some(start) = ctx.data() else {
        return TC_ACT_OK;
    };
    let frame = unsafe { parse::frame(start, ctx.data_end()) };

    // Parse the composite key; None means non-TCP or truncated - pass.
    let Some(key) = parse::allow_key(frame) else {
        return TC_ACT_OK;
    };

    // Fail-open: a missing entry allows. An allowlist that was never
    // populated must not black-hole the interface (Ch.6.3).
    match ALLOW.get(&key) {
        Some(rule) if rule.action == ACTION_ALLOW => TC_ACT_OK,
        Some(_) => TC_ACT_SHOT,
        None => TC_ACT_OK,
    }
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}
}

The program is the XDP shape of Chapter 5 with one crucial difference: it runs on an skb, so everything the stack already did - GRO merging, validation, routing - is behind it, and the socket-level helpers of later chapters (redirect into a sockmap, edit the skb) are available to it. Note also the fail-open choice in the last match arm: an allowlist whose table is empty would otherwise block the entire interface. That decision - what happens when the map is empty, stale, or half-written - is a control-plane contract, and Chapter 14 tests exactly this case.

6.4 Attaching tc from Aya: The clsact Dance

tc attachment requires creating the clsact qdisc first; Aya wraps this but the sequence is worth seeing once, because it is the most common attach failure in the book:

Source: code/ch06_tc_hooks/userspace/src/main.rs

// code/ch06_tc_hooks/userspace/src/main.rs
// Userspace side: create clsact, attach to ingress, fill the allowlist.
use aya::programs::{tc::SchedClassifierAttachMode, SchedClassifier};
use aya::Ebpf;
use std::error::Error;

const KERNEL_ELF: &[u8] =
    include_bytes!("../../kernel/target/bpfel-unknown-none/release/kernel");
const DEFAULT_IFACE: &str = "eth0";
const ALLOWED_IP: u32 = 0x0a00_0001; // 10.0.0.1
const ALLOWED_PORT: u16 = 443;
const ACTION_ALLOW: u32 = 1;

#[repr(C)]
pub struct Rule {
    pub action: u32,
}

fn main() -> Result<(), Box<dyn Error>> {
    let iface = std::env::args().nth(1).unwrap_or_else(|| DEFAULT_IFACE.to_string());

    let mut ebpf = Ebpf::load(KERNEL_ELF)?;
    let program: &mut SchedClassifier = ebpf.program_mut("ingress_check")?.try_into()?;
    program.load()?;
    // clsact must exist before any classifier attaches (Ch.6.4); Aya
    // creates it as part of the attach.
    program.attach(&iface, SchedClassifierAttachMode::Ingress)?;
    println!("tc ingress classifier attached to {iface}");

    let key = (u64::from(ALLOWED_IP) << 16) | u64::from(ALLOWED_PORT);
    let mut allow: aya::maps::HashMap<_, u64, Rule> = ebpf.map_mut("ALLOW")?.try_into()?;
    allow.insert(key, Rule { action: ALLOW }, 0)?;
    println!("allowlist: 10.0.0.1:443 allowed");
    Ok(())
}

The clsact requirement is why tc eBPF demos always show a tc qdisc add dev eth0 clsact step: the qdisc is the hook’s mount point. If you forget it, attach fails with a clear error; if you attach to a veth or a bond, check that the underlying driver supports clsact on that device type.

6.5 Editing the skb: bpf_skb_store_bytes and the Verifier’s Rules

Unlike XDP, a tc program can modify the packet: bpf_skb_store_bytes writes into the skb with the kernel managing the buffer, and bpf_skb_change_proto / bpf_skb_adjust_room grow or shrink it. This is the machinery of NAT, encapsulation (VXLAN/Geneve), and policy rewriting. The verifier’s rules are the price: after a change helper, the old packet pointers are invalid (Chapter 5’s rule, applied to skb), and the checksum helpers (bpf_l4_csum_replace) must be called in the right order. The discipline is identical to XDP - recompute bounds after every mutation - and the consequence of getting it wrong is a packet the kernel forwards with a stale checksum, which is worse than dropping it.

Cilium’s use of these helpers is the reference implementation to study: the bpf_lxc program that implements pod policy edits and redirects is a few hundred lines of exactly this shape - parse, lookup policy in maps, edit if needed, return the action. Chapter 11 walks it.

6.6 The Socket Filter: Where BPF Was Born

The oldest BPF hook is the socket filter: a program attached to a socket that runs on every packet queued to that socket, and whose return value decides how many bytes of the packet the socket sees (return 0 to drop it, return the header length to peek, return the full length to accept). This is the direct descendant of the original 1992 Berkeley Packet Filter of tcpdump: the filter language that grew into eBPF. In Aya it is the SockFilter program type, attached per socket.

Its role today is modest but real: cheap per-socket filtering (drop traffic before userspace sees it), traffic accounting, and the cgroup variant (Chapter 9) that filters every socket in a cgroup at once. The important lesson is historical: the socket filter is where the idea of running a small, verified program inside the kernel’s data path was proven safe enough to generalise into everything else in this book. When someone asks “why does the kernel trust eBPF?”, the honest answer begins “because it has been running packet filters since 1992.”

Hands-On Lab

# 1. Build and attach the tc ingress classifier (Aya creates clsact).
cd code/ch06_tc_hooks/kernel && cargo build --release --target bpfel-unknown-none
cd ../userspace && sudo cargo run --release -- veth0

# 2. Confirm the hook is a sched_cls program on the veth.
sudo bpftool prog list | grep sched_cls
sudo tc filter show dev veth0 ingress    # the classic view of the same hook

# 3. Remove the qdisc and watch the attach contract break.
sudo tc qdisc del dev veth0 clsact       # next attach fails: mount point gone

The discipline: an skb with a populated allowlist gets one hash lookup; an empty allowlist still passes everything (fail-open). Both are visible in bpftool map dump name ALLOW - the map is the program’s visible state.

Summary

• tc is the post-stack hook: ingress before protocol dispatch, egress after the stack has produced the skb, attached via the clsact qdisc.
• The XDP-vs-tc table: drop early, decide late. XDP is pre-protocol and cheap; tc sees routing, sockets and conntrack and can edit packets.
• The tc program is the XDP shape plus skb editing: bpf_skb_store_bytes and friends, with pointer invalidation and checksum rules as the price.
• The clsact qdisc must exist before attach - the classic tc attach failure, wrapped by Aya but worth understanding.
• The socket filter is the ancestor of it all: per-socket, decide how many bytes the socket sees, since 1992.

Next: Part III leaves the packet alone and goes deep into the protocol layer - Chapter 7 is the full TCP/IP receive path, from the ACK in the NIC to the recvfrom in your hand.