Author: Paulo Sergio Camillo — pscamillo@gmail.com · GitHub: pscamillo Subject: Empirical performance characterization of ICICLE v4.0.0 on consumer Blackwell, with a prototype-validated optimization target.
An independent, single-GPU characterization of a public open-source library, shared in case it is useful to the ICICLE community and others bringing GPU ZK stacks to consumer Blackwell. Methodology is empirical before theoretical: every claim is a measurement, and the negative results are reported with the same weight as the positive one.
I profiled ICICLE v4.0.0 (BabyBear NTT) on a consumer NVIDIA Blackwell GPU (RTX 5070, sm_120) and bracketed where time actually goes, end to end and per kernel. Four findings, in order of how surprising they were:
-
ICICLE runs on Blackwell only through PTX-JIT. The shipped v4.0.0 CUDA backend embeds SASS for sm_75–sm_89 (max: Ada) plus
compute_89PTX, but no sm_120 cubin. On the 5070 it works because the driver JIT-compiles thecompute_89PTX to sm_120 at first launch. This is more robust than most of the ecosystem (PyTorch, llama.cpp-based stacks, etc., ship no PTX and hard-fail with "no kernel image is available"), but it carries a first-launch JIT cost and depends on PTX forward-compatibility. -
There is no JIT penalty on field arithmetic. A BabyBear Montgomery-multiply throughput kernel built natively for sm_120 and built as
compute_89-PTX-then-JIT measured within run-to-run noise of each other (1213 vs 1257 Gmodmul/s).ptxasproduces equivalent sm_120 SASS along both paths. So a native Blackwell build is worth doing for latency and robustness, not for kernel throughput on this class of code. -
The NTT butterflies are already at the memory ceiling; the digit-reversal pass is the bottleneck. At 2²², ICICLE's
ntt64runs at 84.7% of DRAM peak andntt32ditat 76.5% — well-optimized, even via JIT. But the single largest kernel isreorder_digits_and_normalize_kernel(39% of NTT kernel time), and it runs at only 46% of DRAM peak, bottlenecked on L2 with a measured global-store efficiency of 4 of 32 bytes per sector (12.5%) — the textbook signature of the scattered writes inherent to digit reversal. -
That headroom is recoverable, and I built a prototype that demonstrates it. On the same GPU, same size, same dtype, a scattered bit-reversal (reproducing ICICLE's exact 4/32 store signature) runs at ~375 GB/s, while a shared-memory-staged coalesced permutation runs at ~780 GB/s — ~2×. Since the reorder pass is 39% of NTT kernel time, a coalesced (COBRA-style) digit-reversal would translate to an estimated ~15–20% end-to-end NTT speedup.
Two of these point to concrete, bounded directions for improvement: a native Blackwell build of the CUDA backend (Finding 1) and a coalesced digit-reversal/reorder kernel (Finding 4), both squarely in the memory-access-pattern domain.
| Component | Detail |
|---|---|
| GPU | NVIDIA RTX 5070 (GB205), sm_120, 48 SMs, ~672 GB/s GDDR7 DRAM, ~48 MB L2 |
| Driver / CUDA | 575.57.08 / CUDA 12.9 |
| Host | AMD Ryzen 7 9800X3D, 128 GB RAM, PCIe 5.0 ×16 |
| OS | Linux Mint 22.1 (Ubuntu 24.04 base), kernel 6.8 |
| ICICLE | v4.0.0 frontend built from source (-DFIELD=babybear); v4.0.0 CUDA backend (ubuntu22-cuda122) |
| Tools | cuobjdump, Nsight Systems (nsys), Nsight Compute (ncu) |
All numbers below are from this single machine. The methodology is "empirical before theoretical": every claim is a measurement, and the negative results (no JIT penalty; butterflies already optimal) are reported with the same weight as the positive one.
cuobjdump on the shipped v4.0.0 backend shared objects shows cubins for sm_75 / sm_80 / sm_86 / sm_89 (Ada is the newest), plus compute_89 PTX, and no sm_120 cubin. The released v4.0.0 CUDA backend assets are CUDA 12.2 builds; there is no CUDA 12.8+ / Blackwell variant.
Empirically, the NTT still runs correctly on the 5070: the driver JIT-compiles the compute_89 PTX to sm_120 on first use. The cost is visible as a first-launch latency (the first NTT call paid a multi-second JIT compile; subsequent calls were warm). The backend also reaches out to Ingonyama's license server (5053@license.icicle.ingonyama.com) by default.
For context, this is a pervasive ecosystem gap — most CUDA stacks ship no sm_120 SASS and no PTX, so they fail outright on Blackwell. ICICLE shipping compute_89 PTX is what makes it run at all. The remaining gaps are the one-time JIT latency, reliance on PTX forward-compatibility, and the absence of official sm_120/sm_100 binaries.
To test whether the JIT path itself costs throughput, I wrote a BabyBear Montgomery-modmul throughput kernel (4 independent ILP chains, correctness-gated) and built it two ways:
- native:
nvcc -arch=sm_120(nvcc emits Blackwell SASS directly) - JIT:
nvcc -gencode arch=compute_89,code=compute_89(onlycompute_89PTX embedded; the driver JITs to sm_120 — mirroring ICICLE's backend path)
| build | throughput |
|---|---|
| native sm_120 | 1212.8 Gmodmul/s |
| JIT compute_89 → sm_120 | 1256.9 Gmodmul/s |
The JIT build was marginally faster — i.e., the difference is run-to-run noise, and there is no codegen penalty from JIT for integer field arithmetic. ptxas lowers the compute_89 PTX to the same quality sm_120 SASS that a native build produces. This matches a prior result of mine on 256-bit modular multiplication being IMAD-throughput-bound at the architectural ceiling regardless of how the SASS is produced.
Implication: a native Blackwell build's value is removing JIT latency and the PTX dependency, not speeding up the arithmetic.
A forward BabyBear NTT at 2²² (mixed-radix) decomposes into five kernel launches: one reorder pass plus two ntt32dit and two ntt64 butterfly stages (radix sequence 32·32·64·64). Per-kernel Nsight Compute profile:
| kernel | share of NTT kernel time | DRAM throughput | verdict |
|---|---|---|---|
ntt64 |
25% | 84.7% of peak | at the memory ceiling |
ntt32dit |
28% | 76.5% of peak | near the ceiling |
reorder_digits_and_normalize |
39% | 46.2% of peak | the bottleneck |
The butterfly kernels are well-optimized on Blackwell — there is little to gain there. The reorder_digits_and_normalize_kernel is the single largest cost and is the one leaving bandwidth on the table. Its Memory Workload Analysis:
- Memory throughput 281 GB/s; L2 hit rate 83% (the working set is L2-resident at this size).
- Global loads: 16.4 of 32 bytes used per sector (~51% efficient).
- Global stores: 4.0 of 32 bytes used per sector (12.5%), across 99.8% of sectors — each 32-byte transaction carries a single useful 4-byte field element. This is the classic scatter signature of digit reversal.
A note on caching: at 2²² the array is 16 MB and fits in the 5070's ~48 MB L2, so this pass is L2-bound rather than DRAM-bound. The inefficiency is one of access pattern (sector utilization), which wastes bandwidth at whatever memory level the working set occupies; for sizes exceeding L2 the same pattern wastes DRAM bandwidth directly.
A separate, practical observation: end-to-end, a single NTT over host-resident data at 2²² is ~74% PCIe transfer and only ~12% compute (the NTT compute itself is ~0.26 ms, ~16 Gelem/s; the H2D/D2H copies dominate). On consumer Blackwell the lever for single-shot NTTs is data residency / batching / transfer overlap, not the kernel — worth stating clearly for users.
To check that the reorder bottleneck is genuinely recoverable rather than intrinsic, I built a focused bandwidth study: three kernels over 2²² 4-byte elements on the same 5070, correctness-gated, then profiled with ncu.
| kernel | global-store bytes/sector | wall-clock throughput |
|---|---|---|
| coalesced copy (ceiling) | 32 / 32 | ~1320 GB/s |
| scattered bit-reversal (= ICICLE reorder pattern) | 4 / 32 | ~375 GB/s |
| shared-memory tiled transpose (coalesced permutation) | 32 / 32 | ~780 GB/s |
The scattered kernel reproduces ICICLE's exact store signature (4 of 32 bytes/sector, flagged identically by ncu), confirming the prototype models the real bottleneck faithfully. The coalesced-staged permutation runs ~2.1× faster with no coalescing penalty.
(At 2²² the 16 MB array is L2-resident, so the absolute figures reflect L2-served bandwidth and exceed DRAM peak; the relative ~2× recovery is the result, and it holds at the DRAM level for larger sizes. ICICLE's reorder is likewise L2-resident here.)
Carrying this back: if a coalesced digit-reversal roughly halves the reorder kernel's time, and the reorder is 39% of NTT kernel time, the end-to-end NTT improvement is on the order of 15–20%. The transpose here is the coalescing primitive; a production digit-reversal drop-in is a COBRA-style kernel (tiled transpose plus index reversal) built on the same shared-memory-staging principle — more index bookkeeping, identical mechanism.
All harnesses are single files, correctness-gated, and were run exactly as below.
Field-arithmetic JIT-vs-native (bench_modmul.cu):
nvcc -O3 -arch=sm_120 bench_modmul.cu -o bench_native
nvcc -O3 -gencode arch=compute_89,code=compute_89 bench_modmul.cu -o bench_jit
./bench_native 2048 ; ./bench_jit 2048
ICICLE NTT harness (ntt_bench.cpp, links the v4.0.0 babybear frontend; backend via ICICLE_BACKEND_INSTALL_DIR):
g++ -O3 -std=c++17 -I.../icicle/include ntt_bench.cpp \
-L"$LIBDIR" -licicle_device -licicle_field_babybear -Wl,-rpath,"$LIBDIR" -o ntt_bench
./ntt_bench 24 # sweep 2^16..2^24
nsys profile --stats=true ./ntt_bench 22 22 # kernel + memcpy breakdown
sudo ncu --kernel-name "regex:ntt32dit|ntt64|reorder_digits" -c 6 --set basic ./ntt_bench 22 22
Recoverable-bandwidth prototype (bitrev_bench.cu):
nvcc -O3 -arch=sm_120 bitrev_bench.cu -o bitrev_bench
./bitrev_bench
sudo ncu --kernel-name "regex:k_bitrev_scatter" -c 1 \
--section MemoryWorkloadAnalysis --section MemoryWorkloadAnalysis_Tables ./bitrev_bench
Versions: ICICLE v4.0.0 (frontend built from source, FIELD=babybear; CUDA backend ubuntu22-cuda122), CUDA 12.9, driver 575.57.08, RTX 5070, Ubuntu 24.04.