Finance bought a three-year Compute Savings Plan to cover the company's GPU training. A month later the team cannot launch the H100 instances it needs: the launch fails for lack of capacity. What went wrong, and how should GPU capacity be secured on AWS?
#The team runs 16 p5.48xlarge instances for most of the year and needs them reliably in one Availability Zone for training. Finance assumed the commitment reserved the machines.
Show answer
A Savings Plan is a billing commitment: you commit to a spend per hour and get lower prices on matching usage. AWS describes Compute Savings Plans as applying regardless of instance family, size, OS, tenancy or Region, with savings of up to 72% on compute. Nothing in that buys capacity.
Capacity comes from reservations: - On-Demand Capacity Reservations reserve instances in a specific Availability Zone. They are charged at the equivalent On-Demand rate whether or not you run instances in them, and Savings Plans and Regional Reserved Instance discounts apply to them. So the usual pattern is a Capacity Reservation for the machines plus a Savings Plan for the price. - Future-dated Capacity Reservations let you request capacity for a start date with a commitment period, during which you cannot reduce the count or duration. - Capacity Blocks for ML reserve GPU instances for a fixed period starting up to 8 weeks ahead, paid upfront.
Also check quotas: On-Demand quotas are set as a number of vCPUs per Region, and a p5.48xlarge has 192 vCPUs, so 16 of them need 3,072 vCPUs of quota before any launch can succeed.
Fix for this team: create a Capacity Reservation (or future-dated one) for 16 p5.48xlarge in the training zone, keep the Savings Plan to discount it, and raise the vCPU quota. Because the reservation bills whether used or not, track its utilisation; an idle reservation is pure cost.
Trade-offs
Reserving capacity guarantees launches but bills even when idle; relying on on-demand availability is cheaper until the day the capacity is not there.
Common mistake
Treating a discount commitment as a capacity guarantee.
Follow-up
The training schedule is only busy nine months a year. How would you combine reservations, Capacity Blocks and on-demand to cover it?