Memory Supply Becomes the Bottleneck for AI Data Center Growth: How Reuse Through CXL Helps
Large AI systems need GPUs, servers, cooling, power and space, but an expanding number of deployment plans are now being limited by a different input: memory. The Center for Strategic and International Studies reports that 50% to 70% of all memory chips produced this year will be consumed by data centers. For operators in the U.S., the market is also exposed to a narrow supplier base and to factory expansion schedules that can require years. In effect, memory can become the item that keeps an otherwise ready AI platform from going into service.
The problem is not only a component shortage. It changes the pace at which infrastructure can be built, raises procurement pressure, and pushes organizations to rethink how existing hardware is used. One response emerging across data centers is to reuse memory removed from retired or decommissioned servers inside newer AI systems. When this approach is treated as part of platform design, three practical benefits can appear.
Three benefits of designing for memory reuse
Operators are finding that reclaimed memory can be useful when systems are planned to accept it. Rather than treating older modules as scrap, platform teams can assign them to defined roles inside an AI infrastructure hierarchy. The following checklist summarizes how that can work.
- Earlier capacity growth: Upgrade cycles often leave large server fleets holding substantial amounts of DRAM. That DRAM can be redeployed through Compute Express Link (CXL) memory expansion devices. Instead of using the CPU’s native memory channels, these devices connect memory over PCIe, letting newer DRAM-based servers draw extra capacity from older memory generations. One expansion controller can host a dozen reclaimed modules and more than one terabyte of memory; some controllers also add in-line compression, which nearly doubles usable capacity.
- Reduced reliance on new fabrication output: Every recovered module represents memory that does not have to be produced through the current supply chain. That means each module recovered in this way avoids competing for fab output, and new memory hardware does not need to be manufactured. The result is reduced dependence on remote fabrication plants.
- Longer system life: Servers contain subsystems that improve at different rates. If only one component ages, a modular design can retire that component rather than the whole machine. Disaggregated architectures allow GPUs, CPUs and memory chips to be replaced on separate schedules. A well-designed server can support several component generations, so memory already in the fleet can remain in service while accelerators are updated.
Compatibility also matters. Reuse is much easier when platforms are designed with memory compatibility from the beginning. If systems support open interconnect standards and reserve layout space for expansion devices, reclaimed modules have a natural place to connect. That avoids one-off adapter projects for each new server generation.
Reusing memory in new AI systems supports adding capacity before new fabs are ready, extending scarce HBM supply, and ending the full-system refresh cycle (Source: Adobe Stock).
Using reclaimed DRAM to extend scarce HBM
AI servers use memory in layers, where speed and cost are traded off. High-bandwidth memory (HBM) sits next to the GPU or accelerator, and one HBM stack can provide over 1 terabyte per second of bandwidth. Conventional dynamic random access memory (DRAM) is far cheaper, and a DDR4 module tops out at 25.6 gigabytes per second. CXL, an open interconnect standard built on PCIe, introduces another layer of expanded and pooled memory while maintaining data coherence across the hierarchy. Older modules fit naturally in that layer.
The value of that layering becomes clearer when HBM is considered separately. Within the data center, HBM is the most constrained memory type, and its supply cannot adjust quickly. The vertically stacked design is more expensive and slower to manufacture than conventional memory. For systems/GPUs designed to use HBM, there is no substitute. Because HBM is scarce, every HBM-equipped accelerator is too valuable to dedicate to tasks that do not require it.
| Memory role | Where it sits | Relevant metric |
|---|---|---|
| HBM | Next to the GPU or accelerator | One stack can provide over 1 terabyte per second of bandwidth |
| Directly attached DRAM | Connected through the CPU’s native memory channels | A DDR4 module tops out at 25.6 gigabytes per second |
| Reclaimed memory through CXL | Connected over PCIe as expanded or pooled memory | One expansion controller can host a dozen reclaimed modules and more than one terabyte of memory |
Because of those differences, supporting services such as caching, data preprocessing and retrieval can run on reclaimed memory, preserving HBM for latency-sensitive training and real-time inference. Matching system sizes to workloads also helps prevent expensive components from being overprovisioned across large server fleets.
Validation remains necessary. Compared with directly attached DRAM, CXL-attached memory introduces some latency, so operators must test configurations against real workloads before committing. Qualifying older modules on new platforms in advance means capacity is available when procurement becomes tight.
Beyond procurement: sustainability and refresh planning
Reusing memory also affects refresh policy. Many refresh cycles retire a whole system when only one component ages. Modular, disaggregated designs allow GPUs, CPUs and memory chips to be replaced on separate schedules. A well-designed server can support several component generations, so memory already in the fleet can remain in service while accelerators are updated.
Separating memory from every accelerator upgrade reduces the amount of new memory operators must buy at shortage prices or wait for because of long lead times. It also lowers electronic waste and Scope 3 emissions linked to manufacturing and sourcing new components, supporting sustainability goals that many operators have set.
FAQ: What this means for AI infrastructure planning
For readers evaluating the practical impact, the following questions connect the memory shortage to deployment decisions.
- Is memory the only constraint on AI infrastructure? No. GPUs, servers and data center capacity may be ready, but missing memory can still block deployment. The shortage also affects conventional DRAM, which every server uses, and the data show an increasingly tight market that is creating shortages in multiple industries.
- Why do reused modules need CXL? CXL memory expansion devices connect memory over PCIe rather than using the CPU’s native memory channels. This lets newer DRAM-based servers draw extra capacity from older memory generations, which is useful when reclaimed modules may not fit the direct-attached architecture of a new platform.
- Does reuse solve the HBM shortage? Not directly. HBM remains the most constrained memory type, and its supply cannot adjust quickly. For systems/GPUs designed to use HBM, there is no substitute. Reuse can stretch scarce HBM by moving suitable workloads onto reclaimed memory.
- What should procurement teams check? Operators should test configurations against real workloads before committing, because CXL-attached memory introduces some latency compared with directly attached DRAM. Qualifying older modules on new platforms in advance means capacity is available when procurement becomes tight.
What to watch next
The immediate takeaway for infrastructure owners is not that memory replaces GPUs, servers or data center capacity; it is that memory can decide which deployments proceed and which stall. As demand continues to pressure the supplier base, teams that validate reclaimed modules, design for CXL expansion and keep component upgrade paths separate may be better positioned to maintain delivery schedules and cost control.
Over a multi-year buildout, these benefits accumulate. Operators that plan for reuse spend less on total infrastructure when supply chains are least forgiving. They also gain more control over timelines, because fewer deployments depend on one allocation from one supplier. Memory supply will eventually catch up with demand. Until that happens, operators and vendors that treat hardware reusability as a primary engineering priority will be the ones able to keep building without disruption.