midnight-proofs GPU prover

A GPU path for the midnight-proofs prover

midnight-proofs proves PLONK circuits with KZG commitments over BLS12‑381. Built with the feature gpu, it can run its heavy arithmetic on a CUDA device. The proof system, the keys, the transcript and the verifier stay as they are.

The device takes the G1 multi-scalar multiplication (MSM), the number-theoretic transform (NTT), its inverse, and the quotient. An accepted quotient call does all of its work on the device: the cosets on the extended domain (a coset of a larger evaluation domain), the numerator, the division by Xⁿ − 1 and the inverse transform back to coefficients. The prover reaches each of the four through a hook: a function that it calls in place of the CPU routine. A hook returns exactly the value that the CPU routine returns, or it declines, and then the CPU routine runs on the unchanged input.

A resident daemon holds the SRS (the structured reference string, the public KZG parameters) and the proving key between requests. By default it frees its device copies of the key and its MSM base tables before each reply and uploads them again for the next request. Before a proof that may use the GPU, the daemon checks the device memory. On a native Linux host, the daemon first activates the device and then reads its free memory. If it gets no reading, it counts zero bytes free. Under WSL2 it uses a leased budget. It compares that figure with its estimate, and individual device calls can still decline. When the request ends it overwrites every witness-dependent device and staging buffer with zeros and confirms the cleanup; if it cannot confirm it, the daemon retires. It verifies every proof before it returns it.

  • Runs on the CPU
  • Runs on the GPU
  • Random draw
  • Witness-dependent data

Try it

The daemon and its client build with cargo. These commands serve the example relation of the crate with a test SRS and make one proof. Build with the feature cuda to route proof phases to a CUDA device. Without a CUDA compiler the same command still builds, and the daemon proves on the CPU. The example circuit is small, below the size from which the device takes work by default, so this run checks the service and not the kernels.

cargo build --release -p midnight-gpu-prover --features cuda
P=target/release

# The socket lives in a fresh directory that only you can enter.
D=$(mktemp -d) && chmod 700 "$D"
S="$D/prover.sock"

# A test SRS is generated in-process. It is insecure: use --srs for real keys.
$P/midnight-gpu-proverd --insecure-test-srs --socket "$S" &

# The socket opens after the SRS and the key are ready. Wait for it.
for i in $(seq 1 120); do
  $P/midnight-gpu-prover-client --socket "$S" health && break
  sleep 1
done
$P/midnight-gpu-prover-client --socket "$S" health || exit 1

# 96 bytes: three canonical little-endian BLS12‑381 scalars. Mode 600.
head -c 96 /dev/zero > in.bin
chmod 600 in.bin
# --no-fallback: fail if the daemon does not serve the proof.
$P/midnight-gpu-prover-client --socket "$S" prove in.bin proof.bin --no-fallback --insecure-test-srs

$P/midnight-gpu-prover-client --socket "$S" stop

In a container

The image builds on a host without a GPU and without a CUDA install, because the CUDA toolkit comes from the builder image. The GPU enters the container through CDI, and the SRS is mounted read-only.

docker buildx build -f gpu-prover/container/Dockerfile -t midnight-gpu-prover:local .

# Generate the CDI spec once, and again after every driver update.
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml

# run-gpu.sh rejects a missing or unreadable spec and a spec whose host paths are missing;
# for WSL driver paths it also rejects a driver directory that does not match the live one.
SRS=bls_midnight_2p17   # the name of your SRS file
docker volume create midnight-gpu-prover-socket
gpu-prover/container/run-gpu.sh docker run -d --name midnight-gpu-prover \
  --device nvidia.com/gpu=all \
  --restart on-failure:3 \
  -v /path/to/srs:/srv/midnight/srs:ro \
  -e MIDNIGHT_GPU_PROVER_SRS="/srv/midnight/srs/$SRS" \
  -v midnight-gpu-prover-socket:/run/midnight-gpu-prover \
  midnight-gpu-prover:local

cargo test -p midnight-gpu-prover --all-features runs the CPU-path tests: admission arithmetic with fake device memory, the protocol, the queue, the fallback labels, the circuit breaker and verify-before-return. Every feature compiles with --all-features on a machine without CUDA.

The full steps are in the crate readme and the container readme.

What the PR changes

The change to the existing prover is small and behind a feature that is off by default. The rest is new code beside it.

The parts of the pull request
Part Path What it is
Hooks in midnight-proofs proofs/src/gpu/ The feature gpu adds the module midnight_proofs::gpu and call sites in plonk/prover.rs, poly/domain.rs, poly/kzg/mod.rs and poly/kzg/msm.rs. With the feature on and no hooks installed, the arithmetic runs on the CPU. The feature also changes two things outside the hooks: the quotient buffers live in owners that zero them on release, and the developer tools withhold assigned values in their output. With the feature off, the stock code runs.
midnight-gpu gpu/ The CUDA kernels behind the hooks. The MSM is a vendored copy of sppark with a record of the local changes that an update must keep. With the feature cuda, nvcc compiles them. Without nvcc the crate builds a stub that declines every device call.
midnight-gpu-prover gpu-prover/ The service library, the daemon midnight-gpu-proverd and the client.
Specification gpu-spec/ 85 EARS requirements (GPU-001 to GPU-085), the design, an executable Lean 4 oracle with theorems, a Quint model of the request state machine, and golden vectors for the Rust tests.
Container and service files gpu-prover/container/ A pinned multi-stage image build, a Compose example and a systemd unit.
Compact benchmark gpu-prover/bench/compact/ A harness that proves real Compact contracts on the CPU and on the GPU path, with its results.

None of the new features is a default. A workspace build without them is the stock prover. With every feature on, the workspace still compiles and its tests run on a machine that has no CUDA toolkit and no GPU, because the kernel crate falls back to its stub.

Features and their tests

Each feature of the pull request, with the test that checks it. A test marked with a CUDA device skips without one; the container records are in TESTED.md.

Features, what their tests check, and the tests
Feature What the test checks Test CUDA device
Hooks behind the feature gpu Hooks that decline every call change no proof byte, and a second install is rejected. proofs/tests/gpu_install.rs install_once_and_decline no
MSM cross-check In a test build, every commitment MSM that the hook serves is compared with msm_pippenger; a hook that returns a wrong point panics at the first served commitment whose point differs. proofs/tests/gpu_msm_crosscheck.rs wrong_msm_hook_fails_the_cross_check no
Stub build Without nvcc, every hook declines and the CPU computes the value. gpu/tests/stub.rs stub_hooks_fall_back_to_cpu no
midnight-gpu kernels The device MSM and NTT return the CPU values; the device quotient matches the stock quotient, coefficient by coefficient, on 10,000 random constraint systems. gpu/tests/device.rs msm_matches_cpugpu/tests/device.rs ntt_matches_cpugpu/tests/prover.rs quotient_fuzz_10000 yes
Readback set The result copies are base-domain transform outputs and, when the device serves the quotient, its coefficients; no extended-domain vector comes back. The MSM partial bucket sums are copied to the host and counted apart. gpu/tests/prover.rs readback_is_base_transforms_and_quotient yes
Admission and resident policy On a native host, a cold request activates the device before it reads free memory. Under memory pressure, admission moves the quotient to the CPU or the whole proof. The default policy asks for an eviction after each proof, and keep skips that request. gpu-prover/tests/qa3.rs cold_admission_reads_free_memory_after_activationgpu-prover/tests/service.rs admission_moves_phases_off_the_device_when_memory_is_shortgpu-prover/tests/service.rs resident_policy_evicts_by_default_and_keeps_on_request no
Daemon Health, prove, stats, evict and stop over the socket; one request served, one waiting, a third gets busy. gpu-prover/tests/daemon.rs round_trip_health_prove_stats_evict_stopgpu-prover/tests/daemon.rs queue_of_one_third_request_is_busy_and_health_answers_during_a_proof no
Container and CDI The launcher refuses a missing spec, a spec with missing referenced host paths, and a mismatched or unconfirmed WSL driver directory. gpu-prover/container/check-cdi.test.sh no
Restart cap systemd stops after StartLimitBurst starts; the container stops after on-failure:3. gpu-prover/container/restart-cap.test.sh no
Client CPU fallback After three connection failures the breaker opens and the next call skips the daemon; with the daemon stopped, the client proves on the CPU and labels the proof cpu-fallback. gpu-prover/src/client.rs absent_daemon_falls_back_and_opens_the_breakergpu-prover/tests/daemon.rs client_labels_daemon_and_fallback_results no
Seeded determinism With a test seed, the GPU proof has the bytes of the CPU proof. gpu-prover/tests/gpu.rs seeded_gpu_proof_equals_cpu_proofgpu/tests/prover.rs seeded_proofs_equal_cpu yes
Erasing owners An owner zeroes its whole capacity on drop and never reallocates; the final quotient limbs are zeroed on release. proofs/src/gpu/tests.rs erasing_owner_zeroes_whole_capacity_on_dropproofs/src/gpu/tests.rs quotient_final_limbs_are_zeroed_on_releaseproofs/src/gpu/tests.rs erasing_owner_never_reallocates no
Device wipe check In check mode, a device proof wipes device regions and bytes and the service stays healthy after it. On the test allocation the device check returns 0 after zeroing, 3 after writes to two 64-bit words and one tail byte, and 0 after the wipe. gpu-prover/tests/gpu.rs zero_check_reads_every_wiped_region_back_as_zerogpu/src/wipe.rs device_count_finds_non_zero yes
Quarantine and pending_erasure() A block whose wipe fails is quarantined, retried, and counted as pending until the retry succeeds. gpu/tests/quarantine.rs failed_lib_wipe_is_quarantined_and_retriedgpu/tests/pending_erasure.rs failed_lib_wipe_is_pending_until_retried yes
Context reset after a failed drain An unconfirmed drain retires the device before the device-owner thread runs anything else. Before the reset the device tries again to zero the quarantined blocks. A successful reset clears the quarantine and counts the quarantined blocks whose zeroing stayed unconfirmed as lost unerased. gpu/tests/drain_teardown.rs unconfirmed_drain_in_a_device_call_retires_the_devicegpu/tests/teardown_lost.rs context_teardown_zeroes_quarantine_before_resetgpu/tests/teardown_lost.rs context_teardown_counts_unerased_quarantine yes
Staged copy drain The test injects an error after a staged upload, with work still queued. The upload runs one drain before it returns. When that drain is not confirmed, the next request for the staging buffer is refused and records one erasure failure. gpu/tests/staged_drain.rs staged_copy_sync_failure_drains_and_holds yes
Waiter across the reset A call that waited for the device while the context was reset declines, its input is unchanged, and the context stays retired. gpu/tests/teardown_waiter.rs a_call_waiting_for_the_lock_across_the_reset_declines yes
Retire on an unconfirmed wipe The service retires when it cannot confirm the erasure. gpu-prover/tests/service.rs an_unconfirmed_wipe_retires_the_service no
Device counters After a device proof, a request withdrawn before its first device call gets the label cpu, 0 device quotient calls and 0 MSMs. A request admitted directly to the CPU reports a device peak of 0 and no wipe of the request before it. gpu-prover/tests/gpu.rs counts_and_peak_are_request_localgpu-prover/tests/gpu.rs a_cpu_admitted_request_reports_no_device_peakgpu-prover/tests/gpu.rs a_cpu_admitted_request_reports_no_wipe_of_an_earlier_request yes
Decline counts A below-minimum MSM shows in the reply as a decline by phase and reason, not as a fallback. gpu-prover/tests/qa3.rs below_min_k_msm_shows_in_the_reply_as_a_decline no
Proof labels A CPU proof is labelled cpu. A proof with a served call and a counted fallback is served as gpu-degraded and fails in acceptance mode. After a degraded admission, a proof with a served device call and zero counted fallbacks is labelled gpu, and its admission reasons stay in degraded_reasons. gpu-prover/tests/service.rs cpu_proof_verifies_and_is_labelled_cpugpu-prover/tests/service.rs a_counted_fallback_is_served_degraded_in_production_and_fails_acceptancegpu-prover/tests/service.rs a_degraded_admission_without_a_fallback_is_labelled_gpu no
Withheld values With the feature gpu, the tested failure texts and the tracing floor planner print <withheld>. The tested forms of a witness marker are absent from two error replies and the daemon log. A marker the caller sends is absent from the replies, the client output and the daemon log. proofs/src/dev/redact_tests.rs gate_failure_output_withholds_valueproofs/src/dev/redact_tests.rs tracing_floor_planner_withholds_valuegpu-prover/tests/b6.rs witness_bytes_reach_neither_reply_nor_loggpu-prover/tests/echo.rs no_caller_value_is_echoed no

The hook contract

Every hook obeys the rules below, which the architecture document states in full. They are what keep a proof from the GPU path equal to the proof of the CPU prover for the same witness and the same random values.

  1. The stock value, or a decline. A hook returns exactly the group element or the vector that the CPU routine returns for the same input. There is no approximate result and no third outcome.
  2. A decline leaves the input unchanged. When a hook declines, the CPU routine runs on the same buffers, with the same content, as if the hook did not exist. A hook can decline for any reason: an unsupported size, too little device memory, a device error.
  3. No randomness and no transcript access. A hook draws no random value and reads or writes no transcript state. If it accepts the call, its return value depends only on the arithmetic input.
  4. Every decline is counted. The device path counts each decline and the daemon reads the counters, so a proof that used the CPU for part of its arithmetic shows up in them.
The hook points and the stock code they stand in for
Hook Stock code in proofs/src Result
G1 MSM commit_msm in poly/kzg/mod.rs, which every commitment calls and which runs msm_pippenger on a decline; also msm_specific in poly/kzg/msm.rs One G1 point
NTT The forward transform on the base domain, best_fft_with_twiddles as called from poly/domain.rs A vector of n field elements, in the caller's buffer
Inverse NTT EvaluationDomain::ifft in poly/domain.rs: the transform at the inverse root, then the scaling A vector of n field elements, in the caller's buffer
Quotient compute_nu_poly with its coset transforms, divide_by_vanishing_poly and extended_to_coeff, as one call from finalise_proof in plonk/prover.rs The coefficients of h(X). The cosets and the numerator stay on the device.

The zero-knowledge argument

A proof from the GPU path has the same distribution as a proof from the CPU prover. After a request, the daemon zeroes and frees every device buffer and every staging buffer that held witness-dependent values, and it takes no further request unless it can confirm that.

Where the randomness is

A KZG commitment in midnight-proofs has no hiding scalar. Zero knowledge comes from random values that the prover writes into the polynomials themselves:

  • the blinding rows of the advice columns, except columns that a circuit declares unblinded;
  • the blinding rows of the LogUp multiplicity and aggregator columns;
  • the blinding rows of the permutation products;
  • two random values at each boundary between quotient limbs.

The blinding rows come from the RNG that the caller passes to the prover. The advice values are written right after synthesis. The others are drawn on one thread before the parallel work that uses them. The limb values come from the operating system RNG. All of this is stock code, and it runs on the CPU.

Why the hooks cannot change the distribution

A hook is deterministic arithmetic. For a given input it returns exactly the value that the CPU routine returns, or it declines and leaves the input unchanged. It draws nothing and does not see the transcript. The prover draws its blinding values and runs every transcript operation on the CPU, in the stock order. Circuit code can draw more randomness of its own on the host, as the foreign-curve chip does. The hooks draw none. For the same witness and the same random values, the transcript, the challenges and the proof bytes are therefore those of the CPU prover.

A caller seed alone does not fix the proof bytes, because the limb values come from the operating system RNG. The Lean oracle of the specification proves the same property for the model: any two devices that obey the hook contract give the same transcript and the same number of random draws.

The skew mask is not a blinder

The MSM backend computes

MSM(s) = MSM(s + r) − MSM(r)

with a fixed public vector r. The identity holds because an MSM is linear in its scalars. The purpose is to balance the bucket loads: a column with many equal values would put most points into a few buckets. The mask is public, so it hides nothing, and the argument above does not rely on it.

What crosses back from the device

The readback from the device is limited to the result point of each MSM, the partial bucket sums of the MSM kernel, the outputs of the transforms on the base domain (n values each, written into the caller's buffer), the coefficients of the quotient, and counters. The bucket sums are an intermediate of the vendored MSM: it copies them to the host for the final accumulation, counts the transfers and zeroes the host copies after use. The stock prover computes the same arithmetic results at the same points. The counters are extra: the daemon collects them for its request report. The numerator stays on the device: the division by the vanishing polynomial and the inverse transform run there, and no vector on the extended domain returns to the host.

What stays after a request

The daemon serves many witnesses, so residue matters. Before it takes the next request it overwrites with zeros, and then frees, every device buffer and every staging buffer that held witness-dependent values. The copies that may stay on the device are proving-key polynomials, which are public. On the host, the feature gpu keeps the quotient coefficients and the blinded limbs in owners that zero their whole allocation on release, on the device path and on the CPU path alike.

The trust boundary

The device, its driver and the host that runs the daemon sit inside the trust boundary, as the host does for the CPU prover. During a request the device holds witness-dependent vectors in the clear, so the design trusts whoever can read device memory while a request runs, as the CPU prover trusts whoever can read host memory. The wipe covers the time after a request: once the cleanup is confirmed, the witness-dependent device and staging buffers of the request are zeroed and freed. If the cleanup cannot be confirmed, the daemon retires. A daemon that crashes before the cleanup can leave witness data in device memory. On the host, the quotient coefficients and limbs are zeroed on release. The decoded witness, the other buffers of the stock prover and the intermediates of the stock quotient path are freed without zeroing, as on any CPU prover. The socket is local and owner only.

The oracle and the checks

The specification exists in forms that check each other. The requirements say what must hold. A Lean 4 oracle states the reference arithmetic and the request state machine and proves theorems about them. A Quint model of the state machine, written from the requirements and the design only, produces traces that are replayed through the oracle. Rust tests run the hooks against vectors that the oracle writes.

The check results need JavaScript and an HTTP server.

Source: gpu-spec/RESULTS.md and gpu-spec/COVERAGE.md.

The theorems are about the models. That the code obeys the hook contract is a separate claim, and the Rust tests carry it: the vectors go through the CPU routine, the stub and the device hook, and faults are injected after upload and after launch.

Benchmarks

Compact program suite

Source: gpu-prover/bench/compact, with the raw results in results/.

Transactions on the Preview network

Source: preview-rtx-5080-laptop.tsv, with the hashes in the benchmarks document.

One large circuit, k = 21, on midnight-proofs 0.8.3

Source: gpu-prover/bench/large-circuit, with the method in the benchmarks document.

Batch proving

midnight_proofs::gpu::batch::prove_batch proves several circuits concurrently in waves sized by device memory, and the batch session of midnight-gpu gives an admitted batch the device backend to itself. After each batch MSM the MSM contexts are evicted and checked, and after each transform or quotient the shared scratch, bounce and read-back buffers go through the wipe path. A proof made in a batch is byte-identical to the same proof made alone with the same randomness. The daemon does not use the batch path: it proves one request at a time, one more request waits in the queue, and a further one gets busy at once.

The table below measures one proof at a time in one process. The benchmark installs the hooks in its own process, without the daemon, registers the SRS and the key with the device layer once, and proves in sequence. The daemon keeps its host SRS and proving key between requests; by default it drops the device-layer registrations after each request.

Resident context, one proof at a time

The benchmark tables need JavaScript and an HTTP server.

Source: gpu-prover/bench/h2h/results/RESULTS.md, with the raw rows in raw/.

Architecture

The layers below run from the client down to the CUDA kernels. The client and the daemon are binaries of the crate midnight-gpu-prover. With the feature gpu, the prover crate gains the hook points, erasing owners for the quotient buffers and withheld values in the developer tools. Built without the feature, it is the stock prover: the CPU code of midnight-proofs, unchanged.

client

Sends a proof request. If the daemon does not answer, the client proves on the CPU.

midnight-gpu-prover

The resident daemon. It holds the SRS and the proving key, decides admission from the computed memory need, wipes buffers and verifies each proof.

midnight-proofs, feature gpu

The stock prover. Its blinding draws and every transcript operation stay here, on the CPU. The hooks draw nothing.

  • G1 MSM
  • NTT
  • Inverse NTT
  • Quotient

midnight-gpu

CUDA kernels. On a machine without a CUDA compiler the crate builds as a stub and every hook declines.

  • sppark MSM, Pippenger buckets
  • NTT and inverse NTT
  • Quotient: cosets, numerator, division, inverse transform

The proving pipeline

A proof is a fixed sequence of steps. The GPU path changes only where the hooked arithmetic runs. The sequence, the random draws and the transcript stay those of the CPU prover.

The steps as text
  1. Key and instances. The prover absorbs the verifying key into the transcript, on the CPU. For an ordinary instance column the transcript takes the length and the values. The stock prover can also take a committed instance column, which it commits with an MSM. The daemon accepts ordinary instances only. Each instance column then goes to coefficients with an inverse NTT. This step draws nothing.
  2. Synthesis, on the CPU. The circuit assigns the advice columns. The last rows of each blinded advice column receive random values from the caller's RNG.
  3. LogUp multiplicities, on the CPU. For each lookup the prover counts how often the selected inputs hit each table tuple. A tuple that fills several table rows shares its count between its usable rows. The blinding values of the multiplicity column come from the caller's RNG. They are drawn before the parallel loop starts.
  4. Commit group 1, MSMs on the GPU. The CPU takes the first differences of each advice and multiplicity column. One MSM for each column then commits them against the SRS bases in delta form, where each base is the sum of the Lagrange bases from its index to the end. The hook returns the same G1 point as the CPU code. The commitments enter the transcript, which returns θ, β, γ and the trash challenge. A trash argument holds constraints that apply only where a selector is on; an unrestricted column, the trash column, absorbs their random combination on the other rows, and the trash challenge is the base of that combination.
  5. Permutation and LogUp columns, on the CPU. The prover computes the permutation products, the LogUp helper and aggregator columns and the trash columns. The blinding values of the products and of the aggregators come from the caller's RNG.
  6. Commit group 2, MSMs on the GPU. The CPU takes the second differences of these columns, and the same hook commits them against the bases in double-delta form, the suffix sums of the delta bases. The transcript returns y, which combines all constraints into one identity.
  7. Transforms and quotient, on the GPU. The committed columns go to coefficients with inverse NTTs. The quotient is then one call. Inside it, the polynomials of this proof go to a coset of the extended domain. At each extended-domain point, the quotient kernel interprets compiled expressions and combines their results with the permutation, LogUp and trash checks. Device kernels then divide by the vanishing polynomial and run the extended inverse transform. Of the quotient, only the coefficients of h(X) return to the host.
  8. Limbs on the CPU, commit on the GPU. The CPU splits h(X) into limbs of n − 2 coefficients. At each limb boundary it appends two random values to the lower limb and subtracts them from the first two coefficients of the upper limb, so h(X) does not change. These values come from the operating system RNG. The limbs are committed with MSMs, and the transcript returns the evaluation point x.
  9. Evaluations, on the CPU. The CPU computes the queried evaluations at x and at its rotations. The proof leaves out those the verifier can compute itself: the evaluations of ordinary instance columns, from the public inputs, and the evaluation of the first quotient limb, from the quotient identity. The CPU writes the rest, and the transcript returns x₁ and x₂.
  10. Opening, first commitment. The CPU combines the polynomials of each point set with powers of x₁, each power truncated to 128 bits. It divides each combination by the vanishing polynomial of its point set and combines the quotients with powers of x₂ into f. Each division discards the remainder. One MSM commits f. The transcript takes the commitment and returns x₃.
  11. Evaluations at z, on the CPU. The opening point z is x₃ truncated to 128 bits. The CPU evaluates the point-set combinations at z and writes the values. Only then does the transcript return x₄.
  12. Opening, second commitment. The CPU combines the point-set combinations and f with truncated powers of x₄ into one polynomial F and computes the opening quotient (F(X) − F(z)) / (X − z). One MSM commits it; the commitment is π. The opening draws no random value: x₁ to x₄ are transcript challenges.
  13. Verify and return, on the CPU. The daemon runs the verifier on the finished proof. It returns the proof only if the verifier accepts it.

A diamond marks a step that can draw random values. Every such step is in the CPU lane. A circuit without lookups, for example, draws no multiplicity blinding. A label between two rows names the challenges that the transcript returns after the row above it.

The NTT butterfly

The number-theoretic transform turns the coefficients of a polynomial into its values at the powers of a root of unity ω, an element of order exactly n. The radix-2 transform shown here is a bit reversal followed by log₂ n stages of butterflies. The animation runs it on 8 points of a small field so that every number can be checked by hand.

The steps as text
  1. Input. Eight coefficients a₀ to a₇ in the field of integers modulo 17. In this field ω = 2 has order 8. The prover works in the BLS12‑381 scalar field, on n = 2ᵏ points.
  2. Bit reversal. The element at index i moves to the index with the three bits of i reversed. Indices 1 and 4 swap, and 3 and 6 swap. After this step each stage combines elements that are close together.
  3. Stage 1, distance 1. A butterfly takes a pair (a, b) and writes (a + w·b, a − w·b). In this stage the twiddle factor w is 1 for every pair.
  4. Stage 2, distance 2. The pairs are two rows apart. The twiddle factors are 1 and ω² = 4.
  5. Stage 3, distance 4. The pairs are four rows apart. The twiddle factors are 1, ω, ω² and ω³, which are 1, 2, 4 and 8.
  6. Output. Row i now holds the value of the polynomial at ωⁱ. The inverse transform is the same network with ω⁻¹, followed by a multiplication by n⁻¹.
The values of the example, modulo 17
RowInputBit reversalStage 1Stage 2Stage 3
03381414
1151561
2446214
312271
45110014
5999611
621737
766121213

This is the order of the stock CPU routine for the NTT and its inverse: bit reversal first, then stages with pair distance 1, 2, 4. The twiddle factors of each domain are computed once and cached.

Before those stages, the coset transform to the extended domain multiplies the coefficients by powers of the coset generator and pads the vector with zeros to the extended size. For the BLS12‑381 scalar field the stock routine then runs the stages in the other order: pairs far apart first, bit reversal last. This lets it skip the work on the zero padding. The way back is one inverse NTT of the extended size, with its single scaling, and then the removal of the coset powers.

MSM bucket accumulation

A commitment is a multi-scalar multiplication: the sum of sᵢ·Pᵢ over all scalars sᵢ of a column and the SRS points Pᵢ. The bucket method replaces most of the scalar multiplications with additions. The animation uses six terms and 6-bit scalars in a toy group of order 61, where every point is a known multiple of a generator G.

The steps as text
  1. Terms. Six scalars sᵢ and six points Pᵢ. The scalars are skewed: five of them are equal. Columns of a real witness often look like this.
  2. Skew mask. A fixed public vector r is added to the scalars: s′ᵢ = sᵢ + rᵢ. The mask spreads the points over the buckets when the original scalars repeat. It is public and hides nothing. Some inputs still load the buckets unevenly.
  3. Window decomposition. Each masked scalar is cut into windows of 3 bits. A 6-bit scalar has a low digit and a high digit, each from 0 to 7. For example, 24 = 3·8 + 0 has the high digit 3 and the low digit 0.
  4. Bucket fill, low window. Each point is added to the bucket that has the number of its low digit. A digit of 0 adds nothing. Bucket j now holds the sum of all points with digit j.
  5. Bucket fill, high window. The same for the high digits, into the right-hand column of buckets.
  6. Running sum. A window needs the sum of j·Bⱼ over its buckets. A sweep from bucket 7 down to bucket 1 gives it with additions only: add the bucket to an accumulator, then add the accumulator to the window sum.
  7. Window combine. The high window sum is doubled three times, once for each bit of the window, and the low window sum is added. The result is MSM(s + r).
  8. Remove the mask. MSM(r) is known in advance, because r and the points are public. Subtracting it gives MSM(s), the same group element that the CPU code returns.
The terms of the example. Point Pᵢ is [p]G in a group of order 61.
Pointpsrs + rHigh digitLow digit
P051232430
P1111444555
P217191012
P3231585973
P4312373947
P5401121315
  • Low buckets: 2 holds P2, 3 holds P3, 5 holds P1 + P5, 7 holds P4. The window sum is [26]G.
  • High buckets: 1 holds P2 + P5, 3 holds P0, 4 holds P4, 5 holds P1, 7 holds P3. The window sum is [46]G.
  • Running sum of the low window, buckets 7 to 1. Accumulator: 31, 31, 21, 21, 44, 0, 0. Sum: 31, 1, 22, 43, 26, 26, 26. All values are multiples of G, modulo 61.
  • Running sum of the high window, buckets 7 to 1. Accumulator: 23, 23, 34, 4, 9, 9, 5. Sum: 23, 46, 19, 23, 32, 41, 46.
  • Combine: 8·[46]G + [26]G = [28]G, which is MSM(s + r).
  • Remove the mask: MSM(r) = [53]G, and [28]G − [53]G = [36]G, which is MSM(s).

A commitment to a full column has n = 2ᵏ input terms, with scalars in the 255-bit scalar field of BLS12‑381. In the stock routine, a term with a zero scalar contributes nothing. The commitments of the opening proof can have fewer terms. The kernel uses more windows, and signed digits, which halve the buckets of a window. It sums the buckets in parallel. The method is the one shown here.

Host and device memory, and the wipe

The daemon keeps public key material between requests. When a request ends, it overwrites with zeros every device buffer and every staging buffer that held witness-dependent values. The animation follows one request, then a second request on a device that another program also uses.

The steps as text
  1. Between requests. The host holds the SRS and the proving key. By default the device holds no copy of the key.
  2. Admission. A request arrives. On a native Linux host, the daemon first activates the device and then reads its free memory. If it gets no reading, it counts zero bytes free. Under WSL2 it uses a leased budget. It compares that figure with its estimate, and individual device calls can still decline. If the estimate fits, the request is admitted to the GPU path. Admission registers the public key on the host and uploads nothing yet.
  3. Commitment MSMs. The witness-dependent work starts with the commitments. For each MSM, a staging buffer on the host holds the masked scalars, and they go to the device. The MSM kernel copies its partial bucket sums back, the host adds them up into the result point, and the host copies of the sums are zeroed at once. The points stay with the prover.
  4. Quotient buffers and key copies. During an accepted quotient call, the backend first builds the extended cosets of this proof. They depend on the witness, except the cosets of ordinary instance columns. It then reuses or uploads the key cosets it needs: the fixed columns, the permutation polynomials and the Lagrange selectors on the extended domain, which are public and the same for every request that uses this key. Only then does it allocate the numerator and its scratch space, evaluate, divide and run the extended inverse transform.
  5. Readback. The quotient returns its coefficients, which still depend on the witness, and the CPU blinds the limbs afterwards. With the result points and the outputs of the base-domain transforms, they are what the stock prover computes too. The daemon also reads counters. No vector on the extended domain comes back.
  6. Overwrite with zeros. When the request ends, the daemon overwrites every witness-dependent device buffer and every staging buffer with zeros. On the host, the coefficients of h(X) are zeroed once they are split into limbs, and the blinded limbs when the proof releases them. The public key copies are not touched.
  7. Free. The zeroed buffers are freed before the daemon takes the next request. If the daemon cannot confirm the erasure, it replies with an error and exits. The other vectors of the stock prover on the host, the decoded witness among them, are freed without zeroing, as on the CPU path.
  8. Resident policy, the default. The daemon frees the key copies and its MSM base tables before each reply and uploads them again for the next request. Between requests it holds no key data on the device.
  9. Resident policy, keep. The other policy is a setting that keeps the uploaded key copies on the device, so later requests reuse them. It suits a device with no other user.
  10. Evict under pressure. With the copies kept, another program takes device memory, and the next request does not fit. The daemon first waits and checks again. If the request is still short, the daemon frees its resident key copies, moves the quotient to the CPU and checks the smaller need. In this example that need fits.

The figure groups the buffers by class. The partial bucket sums are zeroed inside the MSM step. For the other classes it shows the latest point at which each is zeroed: the backend zeroes the staging vector right after its MSM, and the cosets and scratch space before the coefficients return. The heights are not to scale.

The fallback path

Every level has a CPU path: the build, the start of the daemon, each hook call and the client. A hook that declines hands its arithmetic back to the CPU routine. Before a proof starts, the daemon climbs an admission ladder (shown here for a native Linux host), a fixed order of memory checks that ends on the CPU. Whatever path a proof took, it leaves only after the verifier accepts it. The animation follows one hook call, the admission ladder and a client that cannot reach the daemon.

The steps as text
  1. A hook call. The prover reaches an MSM, a base-domain transform or the quotient and calls the hook with the arguments of the stock routine.
  2. The hook declines. The device path cannot serve this call. The hook declines, and the caller's buffers hold what they held before the call.
  3. The CPU computes on the same input. The stock routine runs and returns the value. The prover continues, and the decline is counted. The rest of the proof can still use the device.
The steps as text
  1. Admission, step gpu. On a native Linux host, the daemon first activates the device and then reads its free memory. If it gets no reading, it counts zero bytes free. Under WSL2 it uses a leased budget. It compares that figure with its estimate, and individual device calls can still decline. A check that finds room admits the request at once. In the example the estimate is 50% of the device memory and 58% is free, so the request is admitted to the GPU path.
  2. Step wait. A second request finds the memory short: another program holds 62% of the device, the key copies hold 30%, and 8% is free against a need of 50%. The daemon waits for the other device user, for at most 20 seconds by default, then checks again.
  3. Step evict. If the memory is still short, the daemon frees its resident key copies and checks again. At this step the whole quotient, with its cosets, moves to the CPU, while the MSM and the base-domain NTT stay eligible for the device. The need is therefore smaller: 20% in the example. With 38% free the request fits.
  4. Step cpu. If even the smaller need does not fit, as when another program holds 90% of the device, the proof runs on the CPU. In production mode the request is served all the same.
The steps as text
  1. The daemon is down. The client cannot reach the socket. It proves on the CPU.
  2. Circuit breaker. After 3 consecutive failures within 600 seconds the breaker opens. For the next 600 seconds the client skips the daemon and proves on the CPU at once. A served reply resets the count.
  3. Verify on every path. Whatever path a proof took, the verifier checks it before it is returned.
Where the CPU takes over
Level Trigger Behavior
Build The feature gpu is off, or no CUDA compiler is present The stock CPU prover. No CUDA toolkit is needed.
Start No usable device is found at run time Every phase runs on the CPU.
Call A hook declines or a device call fails The stock routine computes that operation on the unchanged input.
Client The daemon is absent or fails The client proves on the CPU. A breaker opens after 3 consecutive failures within 600 s.

Operations

The daemon detects a CUDA device at run time and listens on a local Unix socket with mode 0600, where it serves only peers with its own user id. It serves one proof request at a time. One more waits, and a third gets busy at once. The resident policy, the admission wait, the memory margin, the thread count and an optional device lock are settings, each with a default in the crate readme. With no usable device the daemon still serves proofs, on the CPU.

Every proof carries a label. cpu means that the device served no call, and cpu-fallback that the client proved in its own process. After at least one served call, the label is gpu-degraded when the request counted a fallback, and gpu otherwise. A hook falls back for a lack of memory, a device error, a fault, an input that is not reduced, or a shape the device does not take. A request-level fallback, such as a hang-up or a lost lock lease, also counts, with the phase request. A degraded admission, which moves phases off the device as the evict step of the ladder does, is not a fallback: with a served device call and no counted fallback, the proof is labelled gpu, and the admission reasons are listed in degraded_reasons. Two declines are expected and leave a proof unmarked: a call below the minimum size, and no device. In acceptance mode the daemon fails a request with a degraded admission or a fallback.

The reply also holds the decline counts of the hook contract by phase and reason, the device peak of the request and the SHA-256 of the proof. The device-call counts, the decline counts, the wipe totals and the device peak are those of the request; a request that is admitted to the CPU reports a device peak of 0 and no wipe of an earlier request. The reply holds no witness value, no key and no seed. A request line that is valid UTF-8 but not a request of the protocol gets the error bad-request with the fixed text not a valid request; the reply does not echo the field name, the value or the path the caller sent. A line that is not UTF-8, and other framing failures, get bad-request with their own fixed text.

A benchmark of the GPU path turns off the client fallback, requires the label gpu, and reads device_calls for each operation to establish what ran on the device.

In a container, the image runs the daemon as a non-root user, mounts the SRS read-only and receives the GPU through the NVIDIA Container Toolkit or the Container Device Interface (CDI). Under systemd, an example unit runs the daemon as a service and restarts it on failure, with at most three starts in ten minutes. While the daemon is down, clients prove on the CPU. The build, the run commands and the unit are in the container readme.

Open items

The pull request is a draft. An independent audit of the backend is in progress, and the items below are not yet in the branch. They land as follow-up commits before the pull request is marked ready for review.

  • Intermediates of the stock quotient. The hook layer keeps the quotient read-back, its check-mode copies and the pieces of h in owners that zero their whole capacity on drop. The intermediates of the stock quotient path, which the decline path and check mode run, are still plain vectors: the coset clones in compute_nu_poly, the resize in coeff_to_extended and the vector of extended_to_coeff.
  • MSM bucket memory. The constructor of the vendored MSM does not zero its bucket allocation. The production wrapper masks its scalars; the tree also holds a test probe that runs an unmasked sparse MSM on a reused context. The zeroing at construction waits for a run in this tree that shows an unwritten slot.
  • Oracle citations. Two citations from a requirement to a theorem are too strong. GPU-078 cites acceptance_degraded_refused without its confirmed-wipe condition, so a request can finish with a fallback and a wipe error and get no degraded refusal. GPU-058 cites pin_wsl_budget for replacing both memory readings, but its input exercises only the first admission branch.
  • Async wipe release. A block that is zeroed, has its free enqueued and then fails the stream synchronisation is neither quarantined nor visible in pending_erasure().
  • Unsafe-code audit. Items UNSAFE-22, UNSAFE-23, UNSAFE-28 and UNSAFE-31 were open in the audit of the unsafe code and are not yet checked against this branch.

Coverage gaps

The coverage matrix marks 6 requirements as Gap: GPU-001, GPU-014, GPU-015, GPU-019, GPU-051 and GPU-065. It marks 8 as Partial: GPU-002, GPU-004, GPU-022, GPU-025, GPU-033, GPU-045, GPU-047 and GPU-053. The matrix is in gpu-spec/COVERAGE.md.

Other notes

  • A drain that fails. When a recorded drain cannot confirm a stream, the CUDA core keeps the affected staging and read-back buffers and hands out no new host buffer. The device-owner thread then, before it runs anything else, disables the device, tries again to zero the quarantined blocks, and attempts one reset of the CUDA context. A successful reset ends every transfer, clears the quarantine and counts the quarantined blocks whose zeroing stayed unconfirmed as lost unerased. A failed reset leaves the device disabled, keeps the quarantine records and leaves the unconfirmed drains outstanding. Every later call declines; a call that reaches the retired-device check records teardown, and calls queued for the device when the teardown starts decline with teardown unrun. No test yet covers a failed reset. pending_erasure() counts the unconfirmed drains until a successful reset settles them. The library never exits, and the daemon does not read pending_erasure() yet.
  • Static teardown. The device list of the vendored code lives behind a pointer that is never destroyed, and its NTT tables are owned and freed by teardown_ntt_tables() on the device-owner thread. This removes the exit-time destructors of the vendored code. The daemon does not call the table teardown yet; at exit the driver releases the tables.
  • Error paths. Failed staged and direct copies run an error drain before they return. Failures of a device wipe or of a release have their own erasure accounting. The MSM error arms of the vendored code drain every stream and stop C++ exceptions at the C entry points. The MSM collector waits for every worker it started, and each worker and the collector scrub a 64 KiB stack region. The device fault tests check the drains, the exception handling and the recovery; the stack scrub was checked by reading the source.
  • The staging size is an atomic value. A device test polls it from a second thread while staged copies run; a data race is not deterministic, so this test is weak evidence. The kernel field arguments are copied with memcpy, and the error path frees the owned message of the vendored code.
  • The quarantine report, pending_erasure(), gives counts and bytes. It counts the blocks of the vendored code whose zeroing is unconfirmed apart from the zeroed blocks that wait for their free, and the device regions of the Rust crate apart from both. It also counts the quarantined blocks whose zeroing was still unconfirmed when a reset succeeded; the live buffers that the reset releases are not in that count. It reports the outstanding unconfirmed drains apart. It does not name a single block. A public buffer whose free failed is quarantined and retried like a witness buffer.
  • With the feature gpu on, MockProver failure text, the debug output of the dev types and the tracing floor planner print <withheld> in place of assigned cell values. With the feature off, they print values as before. The stock floor planners still put Constant(<value>) in assignment annotations outside the dev tools; that is left as a suggestion for upstream.
  • At the pinned versions, the zkir library logs private inputs at error level on malformed-witness paths, under the target midnight_zkir, when a tracing subscriber is installed. The stock proof server filters only zkir. The benchmark wrapper turns off both targets, and the daemon installs no subscriber. The security document gives the operator setting.
  • The stock proof server 9.0.1 binds all interfaces and has no bind option. The benchmark runs it in a loopback-only network namespace behind a Unix socket.
  • Two teardown tests have no run against the code before the fix: the second test in gpu/tests/teardown_lost.rs, and the device test that holders of a device pointer make no CUDA call after the reset.
  • The Quint model was checked by simulation and trace replay. Exhaustive model checking is future work.
  • The new CI workflows have not yet run on a GitHub runner. Their steps pass when run locally.

Repository docs and evidence