RequantTESTNET
Requant research

Verifiable GPU Inference: Batching, Proofs and Memory

Abstract

Can a compact verifiable-inference implementation handle repeated and batched requests within a consumer-GPU budget? Two cohorts retain complete process costs and separately verify every request. All 560 request proofs are accepted under the stated next-token coverage, and batches amortize process/model setup. Observed memory is sampled rather than an instantaneous whole-device peak. The unchanged full-model script stops before GPU execution because its 280 GB available-host-memory requirement exceeds the server budget; no production-model throughput claim follows.

This is the web edition of a completed experimental follow-up, with its original numerical tables, reproduction instructions and limitations. Read the baseline study for the earlier methods and results, and the completed source report for the retained repository edition. The experiments are from November 2026; this web edition was published on 15 November 2026. It is a Requant technical report, not a peer-reviewed publication or independent external replication.

Experimental report

This supplement closes the concrete batching/reuse and resource-qualification questions in the historical report. It uses synthetic weights and the already accepted commitment; the native CPU verifier, compiler sample, arithmetic checks and resource guards remain intact.

Repeated native batch interface

The four profiles are one request, a batch of two (-in-step 2), a batch of four (-in-step 4), and one request with all four documented forest rebuild levels set to five. All use the same commitment and fresh seeded prompts; profiles share their first prompt within an iteration and their order alternates. There are five warm-ups and thirty measured observations per successful profile in each of two seeded cohorts. The model commitment is reused, while each timed call still starts a process and loads its model/key state. This tests native request batching, not a persistent resident server.

The complete generation wall includes process startup, model/key loading and proof generation. CPU verification of every request is recorded separately. All service proofs cover r=63 e=64 T=64, tokens=none, y=yes, next-token=yes, under the declared exact-match acceptance policy; they do not certify that earlier prompt tokens were the model's choices. Each accepted native verifier returns exit code 0 with entire last line YES. Its coverage, commitment assumptions, work and result-v1 lines are retained per request, rather than reduced to a stronger security label.

The caller-supplied commitment ID is 4f351f1f2123409da095d4df1140fcd39a06719d9061163cb62c6719ad2f1943, taken from our own original commitment operation. The existing C++/CUDA implementation has not thereby become a formally proved implementation; the upstream commitment assumptions and proof-of-work accounting remain part of each native acceptance statement.

SeedProfileRequests per batchGeneration median, msp95, msMedian per request, msFirst-probe observed GPU MiB
20261112single13223.023827.323223.024608
20261112batch224075.594679.532037.794676
20261112batch445607.816077.281401.954850
20261112recompute513249.893825.113249.894608
20261212single13279.343979.833279.344608
20261212batch224052.924483.802026.464676
20261212batch445532.216085.451383.054850
20261212recompute513303.723576.623303.724608

All 560 per-request proofs across the two cohorts were accepted by the native CPU verifier.

Batching amortizes the process/model setup. The separate rebuild-five profile gives no material memory reduction at this small workload, so the data do not support changing its default memory policy on that basis. This observation does not predict a production model's throughput or general latency distribution.

Observed memory and full-size gate

The first warm-up of each profile samples GPU0 compute-process allocation through NVML at nominal 100 ms intervals. These probes are excluded from measured observations. This is an observed sampled process allocation peak, not a true instantaneous whole-device memory peak. Native pool counters and sampled allocations describe different boundaries. The production frontend scrubs profiling environment on re-execution; its hardening was preserved instead of changing it to force a profiler through.

The actual unchanged target script was also invoked for target_full_f2 in a fresh work directory. It returned exit code 2 before any GPU phase: its full configuration requires at least 280 GB available host memory; about 59 GB was available. Nothing was created. The raw resource-gate result retains the entire last line and script output. This is a measured host-memory gate, not evidence that the full model fits or fails solely because the GPU has 16 GB. Running past the guard would endanger unrelated server processes and would not be a valid resource-qualified experiment.

Forest fusion decision

Source-derived resource qualification checks the pinned storage formulas against the observed 101,376-byte per-block opt-in limit. The original stored and second-pass fused kernels require 219,136 and 165,888 bytes. Hypothetical 64-column tiles still require 115,712/116,736 bytes and do not fit. At 32 columns the storage formulas give 64,000/92,160 bytes, but that is not an implemented or qualified kernel: the source also fixes warp K-splitting, four-entry lane work and asynchronous copy subdivisions for its original tile. Changing only the width could produce missing arithmetic or out-of-bounds accesses.

Retain the complete-proof-verified generic fallback. The 32-column budget identifies a possible separate kernel redesign; it does not justify enabling that unimplemented path or claiming its speed. The operational compatibility question is resolved by accepted native proofs with the fallback. Production weights and a newly redesigned fused forest are outside these measured results.

Evidence and reproduction

service_study.py defines all profiles, fresh requests, timing boundaries and fail-stop limits. Invoke it with the pinned patched checkout, private synthetic work directory and results/consumer-cycle1.json, then repeat with --seed 20261212 --output results/service-replication.json. Use fresh seed-specific work directories. Public service examples contain only accepted public configuration, verifier key, request, output and proof. No private proving key or failed response/proof is exported.

The unchanged public-artifact replayer checked each of the eight full-cohort examples and seven altered controls per example: all eight honest examples accepted and all 56 altered controls rejected in 64 fresh native calls. verify_followups.py checks every new source/artifact binding, the original binary/commitment reference, all per-request native verdicts, controls and the derived median/p95 table. Hosted CPU CI independently rebuilds the pinned verifier and repeats public-proof controls. CI does not rerun GPU timings.

The completed service decision is to use the supported batch interface when requests can be grouped, retain the qualified fallback and avoid extrapolating the small model to the gated full configuration. Formal security, hiding, arbitrary production weights and heterogeneous multi-GPU operation are not established by native acceptance here.

Server timing boundary

GPU campaigns run serially on the same RTX 5060 Ti. Clocks, power limits and unrelated server services are unchanged. Measurements describe this server and retained workload; small percentage differences are not treated as universal improvements. No energy measurement or heterogeneous multi-GPU throughput is inferred.

Cite this report

Requant. Verifiable GPU Inference: Batching, Proofs and Memory. Technical report, 15 November 2026. Canonical web edition. Experimental evidence revision: 4b784115420b. No DOI has been assigned.