AAperture
Back/ai ecosystem

AI Inference Rationing — The Compute Wall, 2026

Updated 2026-04-21
3 min read
743 words

AI Inference Rationing — The Compute Wall, 2026

Per-token prices fall. Total AI spend rises. Agent workloads consume 10–50× more tokens than chat. The subscription math always broke down at scale; agents made the break visible.

The Core Tension

Training compute is a one-time cost. Inference is a recurring tax. Inference now represents roughly two-thirds of total AI compute demand. A $20 flat subscription priced for chat cannot cover an agent running coding loops for 8 hours a day.

The binding constraints are not GPUs:

  • Power — US data centers consumed 4.4% of national electricity in 2023, projected 6.7–12% by 2028. New generation and transmission take 5–10 years. Labs are turning to gas turbines, nuclear restarts, and behind-the-meter generation.
  • Grid and permitting — ~40% of planned 2026 US AI data centers face delays or cancellations. Transformers on 12-month backorder.
  • Supply chain — NVIDIA lead times 36–52 weeks.
  • Cooling and land — Liquid cooling mandatory; gigawatt campuses need industrial water access.

Money cannot substitute for grid capacity.

Lab-by-Lab Status (April 2026)

Anthropic — Wall already hit

OpenClaw subscription access cut on April 4, 2026. Opus 4.7 rate limits tightened, then partially walked back after user backlash. /usage telemetry rolled out for self-policing. KYC verification rolling out. Business revenue reportedly doubled February–April to ~$30B run-rate.

Anthropic rents inference from AWS and GCP. Vertical integration is absent. No amount of disciplined execution rewrites the cost curve.

OpenAI — Wall projected H2 2026

Same pattern, delayed. Codex quota complaints appeared weeks after Claude's. Sora was cancelled outright (not gated) to conserve compute. Stargate adds 5–10 GW but not until 2028–29; bridge capacity is rented through 2026. The Abilene data center cancellation shortened the runway.

Tighter rate limits, usage dashboards, and KYC are the expected next steps.

xAI — Wall projected 2027–28

No visible rationing as of April 2026. xAI merged with SpaceX in February; Terafab (Tesla + SpaceX + xAI, $20–25B) targets custom silicon in 2027–28. Tesla AI5 taped out late; AI6 targets December 2026; production chips before 2027–28 are unlikely. Until then, runs on NVIDIA at Colossus.

Wildcard: Dojo 3 pivoted to space. A SpaceX merger opens a path to orbital data centers with solar power, radiative cooling, and no grid permits. Nothing is in production.

Google — Wall not visible

TPU vertical integration carries unit costs rented-silicon labs cannot match. AI Overviews serves 1B+ queries. Gemini, NotebookLM, AI Studio remain free daily-driver products. There is no demand curve that breaks a vertically-integrated stack at this scale.

Pricing Model Implications

Four pricing patterns that survive the agent era:

  1. Capacity-gated free tier — cheap models only; upgrade gates better models (Cursor, Perplexity, Replit)
  2. Fixed plan + metered overage — the dominant agent-era pattern (Claude Max, Cursor, Copilot)
  3. Outcome pricing — per shipped PR, per report, per qualified lead
  4. Embed inside a platform — partner absorbs the cost (Google, Microsoft, Apple bundles)

Chat-era SaaS could subsidize free tiers with cheap servers. Agent-era cost scales with tokens × sessions × context depth. A single power user can cost more per month than an average user pays in a year.

Infrastructure as Competitive Moat

Vertical integration is decisive at scale. Google owns the silicon (TPUs), the data, and distribution (Android, Chrome, Search, Workspace). This is the cheapest long-term cost curve in the industry.

Anthropic has strong products and weak infrastructure leverage. OpenAI has the biggest brand and the worst infrastructure execution. xAI has the fastest build velocity and the moonshot path (orbital data centers), but today runs on rented silicon.

Per-token cost will continue falling. Total spend will continue rising. Both are true and not contradictory. See product-trends/token-optimization-economics for the optimization side.

Practical Implications for Builders

  • Orchestration and harness logic in code; LLM calls only where reasoning is required
  • Smaller specialized models for narrow tasks — often 10–100× cheaper
  • Context management: /compact, /clear, session splitting, aggressive caching
  • Retries, context size, and parallelism amplify tokens faster than model choice does
  • Enterprise budget framing: workload budgets, not flat-rate seat licensing; heaviest 10% of users drive 60–80% of spend

Sources

Linked from