GPU prices have jumped this year, underscoring how far the market for AI chips remains from any kind of post-boom normalization.
Loading audio narration…
In an interview, Carmen Li, chief executive of Silicon Data, said the firm’s pricing data shows broad increases across older and newer Nvidia GPUs, used to train and run AI models.
The firm’s Neo Cloud H100 index rose from 2.20 to 2.64, a 20% gain over the past three months. The Neo Cloud B200 index climbed from 4.40 to 5.35, a 22% jump. And the Hyperscaler H100 index increased from 7.26 to 7.46, up 3%, according to Silicon Data.
The numbers suggest that demand for AI compute still outpaces supply. Li said the market has not followed the usual pattern in which prices rise when a new chip is first released, then gradually ease as supply improves. Instead, she said pricing for B200s has remained elevated and even moved higher, a sign that capacity constraints are still biting.
Li argued that pressure from this demand-supply imbalance shows across the AI stack: chips, memory, power, and data center space. That constraint has helped keep prices firm even for older hardware such as Nvidia’s H100s, which remain central to AI training and inference workloads.
“The price is going pretty nuts,” Li told me in a recent interview, referring to H100 rental prices.
Cloud pricing power
The largest GPU rental price premium shows up in hyperscaler environments, where customers pay more for the convenience, existing integrations, and access to capacity offered by the world’s largest cloud providers.
H100s cost almost three times as much to rent from hyperscalers such as Amazon, Microsoft, Google, and Oracle, versus neocloud providers like CoreWeave, which run specialized AI services, according to Silicon Data.
Li worked as global head of strategic alliances at financial data giant Bloomberg for more than two years, overseeing enterprise data deals. She left in 2024 to start Silicon Data, specifically to track GPU prices. She aims to make the GPU rental market more transparent. Li said the company ingests hundreds of thousands of pricing points globally and normalizes them into indexes that can be used to understand both spot rentals and longer-term value.
Not much depreciation
This pricing information is crucial as tech giants, AI labs, and other companies spend trillions of dollars buying GPUs and building data centers to power the generative AI boom. One concern is that if GPU prices fall due to weak demand, AI cloud giants may have to depreciate these assets faster, which could hammer their earnings.
So far, the opposite is happening. GPU prices are rising, and even older GPUs aren’t depreciating much, according to Li, who also runs a business called Compute Exchange that resells refurbished GPUs.
In the second year, you can sell a refurbished H100 for 85 cents on the dollar, Li told me. And then in year three, you can sell the same GPU for 84 cents on the dollar.
“My car depreciates a lot faster than that,” she said.
For the moment, AI demand is off the charts, and the supply of compute can’t keep up. That has consequences not only for infrastructure budgets, but also for the economics of AI products that depend on rented GPUs to generate AI tokens at scale.
Sign up for BI’s Tech Memo newsletter here. Reach out to me via email at abarr@businessinsider.com.
