Federated Infrastructure

The GPU Shortage Is Already Here. Your Waste Decides When Your Business Feels the Impact.

November 18, 2026
8 minutes READ
The shortage is not coming

Demand for AI tokens is growing about 10 times a year. The supply of inference capacity is growing about 3.4 times a year. Those are Epoch AI’s estimates, and its researchers conclude that a compute crunch is plausibly near, “if not already here.”

The largest buyers are already saying so. Amazon’s CEO told investors this year that the company will fall short of demand in 2026 despite roughly $220 billion in capital spending, and expects the same in 2027. Memory makers have reportedly sold their entire 2027 output of the high-bandwidth memory that AI accelerators depend on.

That changes the question for every business with an AI plan. GPU capacity is no longer something to provision on request. It is a scarce, rising-cost resource, and the companies that waste the share they hold will feel the shortage first and pay the most for it.

Supply is growing fast, and it is still not enough

The industry is not standing still. Epoch AI estimates that global AI chip capacity has grown about 3.3 times a year since 2022, doubling roughly every seven months. The problem is that every layer beneath the GPU is now running at or near its limit, so supply cannot accelerate to match demand.

ConstraintCurrent positionHorizonSource
Advanced packaging (CoWoS)Supply “remains tight”; TSMC weighing price rises of up to 20%Capacity planned to roughly double in 2027TrendForce
Memory (DRAM and HBM)2027 output reportedly sold out at Samsung, SK hynix and MicronThrough 2027Digitimes, via PCGamesN
Leading-edge wafers (TSMC N3)Approaching full utilizationThrough at least 2027J.P. Morgan Research, via Apollo
US data center power57 GW shortfall; about 33 GW uncovered even with on-site alternatives2026 to 2028Morgan Stanley, via AIMag
Power equipmentGas turbines nearly sold outThrough 2029Apollo

Some analysts expect pressure to ease after 2028, when new data center space may outrun the chips available to fill it. That makes the next two years the period of greatest scarcity, and the period in which today’s infrastructure decisions will be tested.

Waste puts you at the back of the line

Most organizations already hold more GPU capacity than they use. Cast AI’s analysis of production Kubernetes clusters found average GPU utilization of 5%. A VentureBeat Research survey found that 86% of enterprises running their own GPUs operate them at 50% utilization or less.

In a shortage, that idle capacity is the cheapest GPU supply a company can find. Global AI chip capacity doubles about every seven months, so an organization that doubles its useful GPU utilization gains the equivalent of seven months of the world’s chip production, without waiting for a single new GPU. Moving from the 5% Kubernetes average to 40% is worth about 21 months.

Illustrative Quali model. Demand starts at 20% of GPU capacity and grows 3x a year, well below Epoch AI’s industry estimate of about 10x. Useful utilization: 30% with request-driven provisioning, 70% with a control plane.

The two teams in the chart own the same hardware and face the same demand. The team that provisions on request runs out of usable capacity in about four months and starts buying at scarcity prices. The team that manages its capacity runs out in about fourteen. Better management does not end the shortage; demand is growing too fast for that. It decides who feels it first and who pays the most.

Scarcity reaches the P&L in four steps

A shortage at the chip level does not stay there. Each layer passes its higher costs to the next, until they arrive in development budgets and production margins.

  1. Components cost more. TSMC is reportedly weighing packaging price rises of up to 20% and wafer price rises of 3 to 6% from January 2027. Spot DRAM prices are up about eightfold since early 2025.
  2. Capacity costs more. The neocloud H100 rental index is up 53% from its low late last year, one-year H100 contracts are up 38% year over year, and AWS raised H200 Capacity Block prices by 15%.
  3. Cloud costs more. Gartner expects hyperscalers to recover AI infrastructure costs by raising prices on traditional cloud services. It also warns that companies that fail to optimize their compute could pay up to 50% more than leaner competitors.
  4. AI projects cost more than planned. Gartner predicts that through 2027, enterprises using uncapped, usage-based token pricing for IT operations agents will exceed initial cost expectations by more than 50%. It also expects half of AI service desk projects to be abandoned by 2027 because of unforeseen costs, risks or missed returns.

Most AI business cases approved in the past year assumed last year’s prices. Every development environment left running and every production cluster sized for peak load now costs more each quarter, and the cost lands on the business owner, not the infrastructure team.

Catapults deliver. Control planes decide.

The last generation of infrastructure tools was built for abundance. Cloud management platforms, infrastructure-as-code pipelines and container autoscalers were designed to launch what was asked for, as fast as possible. They work like catapults: they measure success by delivery, not by what the resource achieves once it lands.

Kubernetes shows the limit. It scales on metrics without knowing the purpose, priority or business value of a workload, so it will expand a runaway model as readily as a revenue-critical one. In a world of unlimited capacity that was a cost problem. In a shortage, it takes GPUs away from the work that matters.

Scarcity requires a different kind of system, one that assumes resources are finite and decides how to use them.

QuestionCatapult provisioningInfrastructure control plane
Starting assumptionCapacity is available on requestCapacity is finite and contested
Who gets GPUsWhoever asks firstWorkloads ranked by use case, priority and policy
Idle capacityStays allocated until someone noticesReclaimed by time limits, schedules and cost policies
Spending limitsReported after the month closesEnforced at deployment; environments over a cost threshold are paused for review
AI agentsAct with whatever credentials they holdAct inside the same quotas, approvals and audit trail as people
Measure of successSpeed of deliveryValue delivered per GPU-hour

Quali built Torque on the second model. It tracks every accelerator, reserves and schedules GPU capacity, applies quotas across GPU, CPU and cost at once, shares live environments across teams instead of duplicating them, and pauses environments that exceed their cost policy. The decisions follow the workload and its business purpose, not the order in which requests arrive.

Judge the tools by business outcomes, not specifications

Infrastructure tools are still bought on feature lists: supported clouds, provider integrations, template formats. That habit made sense when capacity was cheap. It does not survive a shortage. Gartner found that 79% of technology buyers regretted a purchase, and it now predicts that organizations without a strong infrastructure and operations function driving AI infrastructure will be more than twice as likely to fail in their AI initiatives.

A tool chosen for a GPU shortage should be measured on five business outcomes:

  1. Useful GPU utilization: the share of paid GPU-hours that runs productive work, by team and workload
  2. Idle capacity reclaimed: GPU-hours recovered each month through time limits, schedules and sharing
  3. Cost per environment and per project: visible in real time, attributed to an owner
  4. Spend controlled before it happens: policies enforced at deployment rather than reported after the invoice
  5. Time to a governed environment: how fast a team gets a compliant, ready-to-use stack

Infrastructure is no longer a back-office cost to minimize. In a shortage it decides which AI projects run, how fast and at what margin. The companies that treat GPU management as a business discipline will keep building while their competitors wait in line.

This article is part of Quali’s Federated Infrastructure series, listed here in publication order.

See how Torque provides the control plane for infrastructure automation across cloud, data center and edge environments, or talk to us about how federated infrastructure could apply to your environment.