Tokens per watt usually means token throughput, measured in tokens per second, divided by power draw in watts. Dimensionally, that is tokens per joule. Comparisons are meaningful only when the model, precision, latency target, batch size, and system boundary are held constant.
For a decade the headline spec was FLOPS, raw compute per chip. FLOPS still describe peak throughput, but they say nothing about the constraint that now bounds every hyperscale buildout: available power. A campus can order more GPUs than its substation can feed. Tokens per watt measures output inside that fixed envelope, which is why CIQ calls it "the new CEO metric".
Inference makes energy efficiency an operating constraint. Deloitte estimates that roughly two-thirds of AI compute in 2026 goes to running models (Deloitte).
NVIDIA reports an up-to comparison for GB300 NVL72 running DeepSeek-R1, based on SemiAnalysis InferenceX data:
| System / workload | Throughput per megawatt | Token cost |
|---|---|---|
| Hopper baseline | 1× | 1× |
| GB300 NVL72, DeepSeek-R1 | up to 50× | up to 35× lower |
The figures come from NVIDIA's own accounting. They measure different outcomes and should not be treated as a direct energy-to-cost conversion.
Huang elevated the metric at GTC 2026 as data-center power became a binding deployment constraint. The phrase predates the event, while NVIDIA's framing pushed it further into infrastructure purchasing discussions.
Cross-vendor comparison will require standardized workloads, latency targets, precision, and measurement boundaries.
The Signal is the public edge of a private practice. Sherpa points the same intelligence engine at one owner's business — competitors, suppliers, regulators, watched daily, graded and sourced. Work with a Sherpa →