Chapter 15b: Benchmarking CUDA-Oxide on Jetson Orin

Code companion: the complete, buildable code for this chapter lives in arpanpathak/cuda-oxide-demo. The article version is CUDA_RUST_JETSON_BENCHMARKS.md.

Chapter 14 showed how CUDA-Oxide compiles idiomatic Rust kernels to PTX. This chapter measures those kernels on real hardware and asks when the GPU is worth using on an embedded SoC.

Four kernels are benchmarked on a Jetson Orin:

1. SAXPY - a memory-bound vector operation.
2. Dot product - a block reduction.
3. Matrix multiply - naive versus shared-memory tiled.
4. Jacobi solver for the 2D Laplace equation - a stencil that solves a partial differential equation.

Every GPU result is verified against a CPU reference. The CPU baseline uses Rayon so that all eight ARM cores participate.

15b.1 Hardware Context

The Jetson Orin is not a datacenter GPU. It is an embedded SoC in which the CPU and GPU share:

• the same DRAM bandwidth,
• the same power budget,
• the same thermal envelope.

This changes the performance story. On a discrete GPU, many kernels beat the CPU. On an SoC, the CPU is competitive for memory-bound work because both processors are limited by the same memory system.

The chart shows:

• SAXPY: 1.8x GPU speedup.
• Dot product: roughly parity (0.7x to 1.8x across runs).
• Tiled matmul: 7.0x GPU speedup.
• Jacobi global stencil: 1.6x GPU speedup.

Compute-bound work with data reuse is where the GPU dominates. Memory-bound work is closer to a tie.

15b.2 The Kernels in the Demo

The repository is modular. Each benchmark binary is split into small single-purpose modules:

src/bin/06_benchmark/
├── main.rs        # entry point
├── kernels.rs     # #[kernel] device functions
├── cpu.rs         # Rayon CPU references
├── data.rs        # generators + correctness checks
├── measure.rs     # wall-clock / CUDA-event timing
├── bench.rs       # one small method per benchmark
└── report.rs      # Markdown/CSV output

src/bin/07_laplace_jacobi/
├── main.rs        # entry point + parameters
├── kernels.rs     # global + shared-memory stencils
├── cpu.rs         # Rayon CPU Jacobi reference
├── data.rs        # correctness checks
├── measure.rs     # CPU timing helper
├── bench.rs       # orchestration + GPU event timing
└── report.rs      # results printing + file output

SAXPY

#![allow(unused)]
fn main() {
#[kernel]
pub fn saxpy(alpha: f32, input_x: &[f32], input_y: &[f32], mut output: DisjointSlice<f32>) {
    let index = thread::index_1d();
    let position = index.get();

    if let Some(slot) = output.get_mut(index) {
        *slot = alpha * input_x[position] + input_y[position];
    }
}
}

Dot Product Block Reduction

#![allow(unused)]
fn main() {
static mut SHARED: SharedArray<f32, 256> = SharedArray::UNINIT;
// grid-stride accumulation into local_sum ...
unsafe { SHARED[thread_id] = local_sum; }
thread::sync_threads();

let mut offset = 128;
while offset > 0 {
    if thread_id < offset {
        unsafe { SHARED[thread_id] += SHARED[thread_id + offset]; }
    }
    thread::sync_threads();
    offset /= 2;
}
}

Tiled Matrix Multiply

Each 16x16 block loads a tile of A and a tile of B into shared memory, synchronises, computes the output tile from shared memory, synchronises again, and advances to the next K-tile. This change turns the naive GPU kernel into a roughly 6x faster kernel.

Jacobi Solver

The Laplace equation is the steady-state diffusion equation. On a grid, the discrete form becomes:

u_new[i,j] = (u[i-1,j] + u[i+1,j] + u[i,j-1] + u[i,j+1]) / 4

Each iteration replaces every interior cell with the average of its four neighbours. The top boundary is fixed at 100, the other edges at 0, and the iteration spreads the boundary information inward.

15b.3 Where the Laplace Equation Appears

The Laplace equation \(\nabla^2 u = 0\) describes equilibrium in physics:

• Heat conduction: steady-state temperature in a solid.
• Electrostatics: electric potential in a charge-free region.
• Fluid dynamics: pressure in potential flow.
• Image processing: inpainting, smoothing, and edge detection.
• Graphics: surface fairing and smooth height fields.

The Poisson variant \(\nabla^2 u = f\) adds sources. Jacobi iteration is the simplest solver and is the foundation for the multigrid and Krylov methods used in production solvers.

15b.4 Benchmark Methodology

• CPU: Rayon-parallel Rust, std::time::Instant, best of 5.
• GPU: CUDA events, best of 5, kernel time only.
• Warm-up: 5 GPU launches before timing so that the Jetson’s clocks ramp up.
• Data: deterministic pseudo-random f32 vectors and matrices.
• Verification: every GPU result is compared against the CPU reference with a tolerance.

15b.5 Results

Vector Kernels

Matrix Multiply

Jacobi Solver

Both GPU Jacobi variants match the CPU field exactly after 500 iterations.

15b.6 Lessons

1. Memory-bound kernels are close on SoCs. When the CPU can already saturate the shared memory bandwidth, the GPU adds little.
2. Shared-memory tiling is the GPU’s main advantage here. Matmul jumps from 32 to 203 GFLOPS, a 6.3x improvement over the naive GPU kernel and 7x over the CPU.
3. Shared memory is not always the answer. The tiled Jacobi stencil is slower than the simple global stencil on the Orin because the unified L2 cache absorbs the halo reads. Measure on the target hardware.
4. Warm-up matters on embedded GPUs. Without warm-up launches, the first measured kernel can be 2x slower because of clock ramping.

15b.7 Reproduce

git clone https://github.com/arpanpathak/cuda-oxide-demo.git
cd cuda-oxide-demo

cargo oxide run --bin 06_benchmark
cargo oxide run --bin 07_laplace_jacobi

Release builds:

cargo oxide build -- --release --bin 06_benchmark --bin 07_laplace_jacobi
./target/release/06_benchmark
./target/release/07_laplace_jacobi

Reports are written to benchmarks/benchmark_results.{md,csv} and benchmarks/jacobi_results.{md,csv}.

Sources and Further Reading

• NVIDIA, Jetson Orin Developer Guide: https://developer.nvidia.com/embedded/learn/jetson-orin
• CUDA-Oxide demo repository used in this chapter: https://github.com/arpanpathak/cuda-oxide-demo