Four agents is the ceiling.

NVIDIA Nemotron 3.5 Lightning 30B-A3B, served by vLLM 0.27.1 on one NVIDIA GB10 — a DGX Spark under a desk, 128 GB of memory shared by CPU and GPU. This page plays back one recording made on 16 August 2026. It reads a run file that ships with the site and never contacts the machine.

Aggregate throughput stops climbing at four

Median aggregate throughput was 62.9 tokens a second at one concurrent stream, 113.1 at two, 178.2 at four and 173.6 at eight. Four to eight is −2.6%, and the interquartile ranges overlap — 165.2 to 181.0 at four against 169.7 to 175.4 at eight — so the honest statement is that eight streams buy no measurable throughput, not that throughput falls.

The mechanism is one line of the serving config: max-num-seqs: 4. In the eight-lane recording four lanes receive a first token within 249 ms and four wait between 2.97 and 6.17 seconds for a slot. The number of lanes emitting at once never reaches five.

What the eighth stream buys is queue

Median time to first token goes from 244 ms at four concurrent streams to 3,150 ms at eight — 12.9x — while aggregate throughput does not move.

What this does not measure

No power, energy, temperature, clock or cost figure was measured, so nothing about tokens per watt or per dollar can be said from this run. Answer quality was not evaluated. The 88% median GPU utilisation is a coarse two-second nvidia-smi sample, not an efficiency claim. The client ran on the machine itself over loopback, so no network is inside the time to first token.

Single-stream throughput on this box is not a stable number: across eight prompts at one lane the per-stream decode rate ranged 38.3 to 100.3 tokens a second, and holding prompt and seed fixed for fifteen requests it still ranged 44.6 to 98.6. Two sweeps an hour apart agreed on the shape and differed on the level by about 15%, so the shape is the finding and the absolute numbers are good to roughly 15%.

Bench pins the target tuple this run was measured against · Evals publishes the score cards