If it feels today as if every little thing in know-how is about AI, that’s as a result of it’s. And nowhere is that extra true than available in the market for pc reminiscence. Demand, and profitability, for the kind of DRAM used to feed GPUs and different accelerators in AI knowledge facilities is so large that it’s diverting away provide of reminiscence for different makes use of and inflicting costs to skyrocket. In response to Counterpoint Analysis, DRAM costs have risen 80-90 precent to this point this quarter.
The most important AI {hardware} firms say they’ve secured their chips out so far as 2028, however that leaves all people else—makers of PCs, client gizmos, and every little thing else that should briefly retailer a billion bits—scrambling to cope with scarce provide and inflated costs.
How did the electronics business get into this mess, and extra importantly, how will it get out? IEEE Spectrum requested economists and reminiscence consultants to elucidate. They are saying right this moment’s state of affairs is the results of a collision between the DRAM business’s historic growth and bust cycle and an AI {hardware} infrastructure build-out that’s with out precedent in its scale. And, barring some main collapse within the AI sector, it can take years for brand new capability and new know-how to deliver provide in step with demand. Costs would possibly keep excessive even then.
To grasp each ends of the story, it is advisable know the principle offender within the provide and demand swing, high-bandwidth reminiscence, or HBM.
What’s HBM?
HBM is the DRAM business’s try and short-circuit the slowing tempo of Moore’s Regulation by utilizing 3D chip packaging know-how. Every HBM chip is made up of as many as 12 thinned-down DRAM chips known as dies. Every die accommodates various vertical connections known as by way of silicon vias (TSVs). The dies are piled atop one another and linked by arrays of microscopic solder balls aligned to the TSVs. This DRAM tower—properly, at about 750 micrometers thick, it’s extra of a brutalist office-block than a tower—is then stacked atop what’s known as the bottom die, which shuttles bits between the reminiscence dies and the processor.
This advanced piece of know-how is then set inside a millimeter of a GPU or different AI accelerator, to which it’s linked by as many as 2,048 micrometer-scale connections. HBMs are hooked up on two sides of the processor, and the GPU and reminiscence are packaged collectively as a single unit.
The concept behind such a good, highly-connected squeeze with the GPU is to knock down what’s known as the reminiscence wall. That’s the barrier in vitality and time of bringing the terabytes per second of knowledge wanted to run massive language fashions into the GPU. Reminiscence bandwidth is a key limiter to how briskly LLMs can run.
As a know-how, HBM has been round for greater than 10 years, and DRAM makers have been busy boosting its functionality.
As the dimensions of AI fashions has grown, so has HBM’s significance to the GPU. However that’s come at a value. SemiAnalysis estimates that HBM typically prices thrice as a lot as different varieties of reminiscence and constitutes 50 p.c or extra of the price of the packaged GPU.
Origins of the reminiscence chip scarcity
Reminiscence and storage business watchers agree that DRAM is a extremely cyclical business with large booms and devastating busts. With new fabs costing US $15 billion or extra, companies are extraordinarily reluctant to develop and will solely have the money to take action throughout growth instances, explains Thomas Coughlin, a storage and reminiscence knowledgeable and president of Coughlin Associates. However constructing such a fab and getting it up and operating can take 18 months or extra, virtually making certain that new capability arrives properly previous the preliminary surge in demand, flooding the market and miserable costs.
The origins of right this moment’s cycle, says Coughlin, go all the best way again to the chip provide panic surrounding the COVID-19 pandemic . To keep away from supply-chain stumbles and assist the fast shift to distant work, hyperscalers—knowledge middle giants like Amazon, Google, and Microsoft—purchased up large inventories of reminiscence and storage, boosting costs, he notes.
However then provide grew to become extra common and knowledge middle growth fell off in 2022, inflicting reminiscence and storage costs to plummet. This recession continued into 2023, and even resulted in large reminiscence and storage firms comparable to Samsung reducing manufacturing by 50 p.c to try to maintain costs from going beneath the prices of producing, says Coughlin. It was a uncommon and pretty determined transfer, as a result of firms usually must run crops at full capability simply to earn again their worth.
After a restoration started in late 2023, “all of the reminiscence and storage firms have been very cautious of accelerating their manufacturing capability once more,” says Coughlin. “Thus there was little or no funding in new manufacturing capability in 2024 and thru most of 2025.”
The AI knowledge middle growth
That lack of recent funding is colliding headlong with an enormous increase in demand from new knowledge facilities. Globally, there are practically 2,000 new knowledge facilities both deliberate or below development proper now, in accordance with Information Middle Map. In the event that they’re all constructed, it might characterize a 20 p.c soar within the world provide, which stands at round 9,000 services now.
If the present build-out continues at tempo, McKinsey predicts firms will spend $7 trillion by 2030, with the majority of that—$5.2 trillion—going to AI-focused knowledge facilities. Of that chunk, $3.3 billion will go towards servers, knowledge storage, and community tools, the agency predicts.
The largest beneficiary to this point of the AI knowledge middle growth is definitely GPU-maker Nvidia. Income for its knowledge middle enterprise went from barely a billion within the remaining quarter of 2019 to $51 billion within the quarter that led to October 2025. Over this era, its server GPUs have demanded not simply increasingly gigabytes of DRAM however an rising variety of DRAM chips. The not too long ago launched B300 makes use of eight HBM chips, every of which is a stack of 12 DRAM dies. Rivals’ use of HBM has largely mirrored Nvidia’s. AMD’s MI350 GPU, for instance, additionally makes use of eight, 12-die chips.
With a lot demand, an rising fraction of the income for DRAM makers comes from HBM. Micron—the quantity three producer behind SK Hynix and Samsung—reported that HBM and different cloud-related reminiscence went from being 17 p.c of its DRAM income in 2023 to almost 50 p.c in 2025.
Micron predicts the entire marketplace for HBM will develop from $35 billion in 2025 to $100 billion by 2028—a determine bigger than all the DRAM market in 2024, CEO Sanjay Mehrotra informed analysts in December. It’s reaching that determine two years sooner than Micron had beforehand anticipated. Throughout the business, demand will outstrip provide “considerably… for the foreseeable future,” he stated.
Future DRAM provide and know-how
“There are two methods to deal with provide points with DRAM: with innovation or with constructing extra fabs,” explains Mina Kim, an economist with the Mkecon Insights. “As DRAM scaling has turn out to be harder, the business has turned to superior packaging… which is simply utilizing extra DRAM.”
Micron, Samsung, and SK Hynix mixed make up the overwhelming majority of the reminiscence and storage markets, and all three have new fabs and services within the works. Nevertheless, these are unlikely to contribute meaningfully to bringing down costs.
Micron is within the strategy of constructing an HBM fab in Singapore that must be in manufacturing in 2027. And it’s retooling a fab it bought from PSMC in Taiwan that can start manufacturing within the second half of 2027. Final month, Micron broke floor on what might be a DRAM fab advanced in Onondaga County, N.Y. It won’t be in full manufacturing till 2030.
Samsung plans to begin producing at a brand new plant in Pyeongtaek, South Korea in 2028.
SK Hynix is constructing HBM and packaging services in West Lafayette, Indiana set to start manufacturing by the top of 2028, and an HBM fab it’s constructing in Cheongju must be full in 2027.
Talking of his sense of the DRAM market, Intel CEO Lip-Bu Tan informed attendees on the Cisco AI Summit final week: “There’s no reduction till 2028.”
With these expansions unable to contribute for a number of years, different components might be wanted to extend provide. “Aid will come from a mixture of incremental capability expansions by current DRAM leaders, yield enhancements in superior packaging, and a broader diversification of provide chains,” says Shawn DuBravac , chief economist for the International Electronics Affiliation (previously the IPC). “New fabs will assist on the margin, however the sooner positive aspects will come from course of studying, higher [DRAM] stacking effectivity, and tighter coordination between reminiscence suppliers and AI chip designers.”
So, will costs come down as soon as a few of these new crops come on line? Don’t guess on it. “Basically, economists discover that costs come down way more slowly and reluctantly than they go up. DRAM right this moment is unlikely to be an exception to this basic remark, particularly given the insatiable demand for compute,” says Kim.
Within the meantime, applied sciences are within the works that would make HBM an excellent larger client of silicon. The usual for HBM4 can accommodate 16 stacked DRAM dies, regardless that right this moment’s chips solely use 12 dies. Attending to 16 has so much to do with the chip stacking know-how. Conducting warmth by way of the HBM “layer cake” of silicon, solder, and assist materials is a key limiter to going increased and in repositioning HBM contained in the package deal to get much more bandwidth.
SK Hynix claims a warmth conduction benefit by way of a producing course of known as superior MR-MUF (mass reflow molded underfill). Additional out, another chip stacking know-how known as hybrid bonding may assist warmth conduction by lowering the die-to-die vertical distance primarily to zero. In 2024, researchers at Samsung proved they might produce a 16-high stack with hybrid bonding, and so they prompt that 20 dies was not out of attain.
From Your Web site Articles
Associated Articles Across the Net
