AI Is Resetting the Memory Stack: Macronix, MemFlation and the Search for Tiers Beyond HBM

For the first wave of AI infrastructure reporting, the bottleneck had a familiar name: accelerators, and the high-bandwidth memory stacked beside them. The consequences are now surfacing further down the bill of materials, in the low-capacity parts that boot a server, hold firmware and buffer edge workloads. Boot code, baseboard management controllers, routers, industrial PCs and automotive systems all draw on NOR flash, SLC NAND and eMMC — and that is where supply has thinned fastest. Speaking with EE Times Taiwan, Martin Lin, director of marketing at Macronix International, put the moment in unusually broad terms: this is not only a demand story, because “the entire memory architecture is changing.”

Below, AIdataNewsGlobal unpacks the numbers behind that claim, the product lines being squeezed, the new memory tiers being drafted to sit underneath HBM, and what procurement and platform teams should still treat as unproven.

Five points that frame the story

• Capacity and capital spending at Samsung, SK Hynix and Micron have been redirected to high-bandwidth memory (HBM) and advanced server memory, leaving mature and specialty products with less headroom.
• Macronix lifted 2026 capital expenditure to NT$37.8 billion to expand 12-inch wafer capacity for 3D NOR, NAND, eMMC and adjacent products.
• Reported market value passed $20 billion in 2025 and may approach $70 billion in 2026, with growth coming increasingly from price rather than shipment volume — a dynamic Macronix labels “memory inflation,” or “MemFlation.”
• The combined segment of NOR flash, SLC NAND and low-capacity eMMC (≤32GB) is expected to reach around $24 billion in 2026.
• Tiers below HBM are moving from concept to product: high-bandwidth flash, high-IOPS server SSDs and key value (KV) cache offloading, the last of them demonstrated by Macronix at the Future Memory and Storage conference (FMS 2026) in the United States.

Why a normal cycle is the wrong frame

Conventional memory downturns resolve when channel inventories normalise. This upturn has a different mechanism. AI build-outs are absorbing the best available wafer starts and backend capacity, and reported bookings at the leading-edge end extend well into 2027. NAND is repricing in parallel as AI data centers and enterprise solid-state drives (SSDs) claim more supply.

Layered on top is a deliberate change in product mix. As majors pull out of older process nodes and move those lines toward higher-margin work, the flexibility to schedule NOR flash, SLC NAND and eMMC narrows. That is why independent industry coverage has increasingly described the shortage as structural rather than seasonal.

The arithmetic follows the capacity shift. Structural changes in supply and manufacturing have lifted memory prices sixfold over the past year, and the pricing curve has not flattened. NOR flash, SLC NAND and eMMC are expected to move higher again in the third and fourth quarters of 2026, then hold at elevated levels.

Relief is slow on the supply side even when money is committed. Equipment installation, process qualification and yield ramp-up can take two to three years once new capacity exists on paper, so the gap cannot be closed by announcements alone. Lin's read is that as AI demand keeps crowding out mature-process capacity, automotive-market supply constraints and pricing pressure should become more visible starting in 2027.

Not all of the specialty portfolio tightens equally. NOR flash faces relatively moderate price increases, helped by capacity decisions taken earlier, while SLC NAND and eMMC remain under heavier supply pressure. Higher shipments plus scarcity-driven pricing are what push the combined category toward around $24 billion in 2026.

The quantity signal: more flash sockets per AI rack

One of the clearest indicators is not megabytes but component counts. NOR flash devices in a server perform boot-up, firmware storage, system control and security roles — functions that multiply as boards gain management silicon.

Conventional servers typically required only about three to five flash devices, Lin noted. AI systems are a different bill of materials: baseboard management controllers (BMCs), network switches and additional control components each add their own boot and firmware storage, so flash consumption per rack has risen rapidly.

Lin said, “The previous-generation rack had around 600 to 800 devices. The latest generation now has more than 1,000”. In other words, AI is buying legacy flash in volume as a side effect of buying accelerators — a change in quantity that has its own market consequences.

After HBM: the search for capacity at a survivable cost

Training and modern inference both lean on HBM for the bandwidth GPUs require. As model and context sizes grow, however, HBM capacity and cost become limiting rather than enabling.

Lin said that “only HBM is fast enough” today. Adding gigabytes and stacking more HBM layers raises cost with each step, so the industry is looking for memory tiers beneath HBM that trade a little speed for much greater capacity and better cost efficiency.

High-bandwidth flash (HBF) is one candidate. Built on 3D NAND and paired with high-speed interfaces and high-density packaging, it attempts to balance bandwidth, capacity and cost, letting data that does not need to stay resident in HBM be pushed to higher-capacity, relatively lower-cost NAND. In parallel, architects are evaluating AI server designs that rely on SSDs delivering a high level of input/output per second (IOPS), where low-latency SLC NAND helps absorb the random-access traffic GPUs generate and improves utilisation.

Inference is the sharper pressure point. Large language models (LLMs) and agentic workloads must continuously retain conversation, task and historical state, so KV cache keeps growing across turns. Citing continuous AI conversations, Lin warned that if memory capacity is insufficient, “you have to discard the earlier information.” That is the failure mode tiering is designed to avoid: recomputation and truncated context instead of retained state.

The emerging pattern is a policy-driven hierarchy rather than a single device class:

1. Hot, latency-sensitive data stays in HBM.
2. Warm context and frequently revisited blocks move to flash and SLC tiers.
3. Bulk, delay-tolerant state lands on SSDs and other capacity layers.
4. Data is placed by age, access frequency and latency requirement instead of by convenience.

At the recent Future Memory and Storage conference (FMS 2026) in the United States, Macronix presented a dual-tier SSD architecture aimed at KV cache offloading, exploring how different memory and storage layers can divide AI workloads. Offload paths of this kind are no longer confined to vendor slides: open-source inference engines have begun shipping tiered context management that spills evicted KV blocks to host memory and NVMe storage rather than recomputing them, which suggests the architectural assumption is becoming shared practice.

The constraint has moved from compute to data movement

Accelerator performance keeps climbing, and the binding limit is shifting to whether data arrives in time. Lin put it bluntly: “The GPU is already fast enough, but now , memory is actually too slow”.

The cost is paid in shuttling. Repeatedly moving data among the CPU, GPU, HBM, DRAM and storage devices consumes bandwidth, adds latency and raises power draw, which is why simply raising processor speed does not translate into system-level gains.

Two older responses are being revisited: memory-centric AI and processing-in-memory (PIM). Both bring computation closer to where data rests, and in some implementations let memory perform simple operations before returning results, cutting the number of transfers.

“Memory-centric AI” is not a new concept. Macronix Chairman and CEO Miin Wu proposed it as early as 2019. Today, as AI models, datasets and inference workloads continue to grow rapidly, the idea is again drawing attention from industry and academia.

How the product roadmap maps to the thesis

Macronix describes this as the biggest structural shift in the memory industry in decades, and its response is to push NOR flash, NAND and eMMC toward denser, lower-power and more reliable designs while adding wafer capacity.

Currently 48-layer and 96-layer 3D NAND products are in mass production, with higher-layer-count technologies in development. NOR flash is moving toward higher density, lower power consumption and greater reliability, with process technology progressing from 55nm to 42nm and 3D NOR also under development. High-density NOR is already shipping into AI servers, while the company expands into high-reliability automotive, communications, medical and aerospace applications. To address power requirements in AI, industrial and embedded systems, low-voltage flash is progressing from 3V and 1.8V toward 1.2V.

Macronix’s three major flash product lines continue to evolve toward higher density and greater capacity.

Line	Direction of travel	Nodes and milestones
NOR flash	Higher density, lower power, greater reliability	55nm to 42nm; 42nm in 2027; 3D-32-layer in 2028
SLC NAND	Long-term supply support	Low-latency role in AI random access
eMMC	Higher capacity and density	3D-48 and 96 layers to 192 layers

Three signals to track into 2027

• Qualification lag. Watch how quickly announced 12-inch capacity clears process qualification and yield ramp-up; supply relief that requires two to three years cannot stabilise pricing in the interim.
• Automotive allocation. As AI demand keeps crowding out mature-process capacity, order books at car-parts suppliers are an early indicator of whether 2027 constraints bite outside the data center.
• Tier adoption. The durability of new tiers below HBM depends on whether KV cache offloading and HBF move from demonstrations and open-source prototypes into production deployments with measurable cost-per-token effects.

Looking toward 2027, Lin expects tight memory supply and demand conditions to remain difficult to ease in the short term, with extra time still required for qualification and yield improvement even as capacity arrives. For Macronix, the balancing act is demand from AI servers, supply gaps in mature flash products and next-generation technology development at the same time.

Frequently asked questions

Is this a price story or an architecture story?

Both, and they reinforce each other. Prices are up sixfold over the past year because capacity was reallocated to AI-facing products. The architectural change is that memory is being reclassified from a passive storage component into a factor that determines AI system performance, cost and design.

Why would AI servers need more NOR flash rather than more DRAM?

Because of control-plane complexity. Boot, firmware, system control and security functions each need flash, and AI racks add management controllers and switches that conventional servers did not carry — hence the shift from three to five devices per box to well over a thousand per rack generation.

Does HBF replace HBM?

No. HBF targets capacity and cost efficiency at a lower latency tier, intended to hold data that does not need the always-on speed of HBM. Lin's framing remains that only HBM is fast enough for the hot tier.

What is MemFlation?

Macronix's shorthand for “memory inflation”: a period in which category revenue rises mainly because supply is constrained and prices increase, rather than because unit shipments alone are doing the work.

How much uncertainty is still baked into these figures?

Meaningfully. The $70 billion and around $24 billion projections are company-supplied market reads, and the pricing path in the third and fourth quarters of 2026 assumes AI infrastructure spending holds. Equipment lead times and qualification cycles, not demand forecasts, are the variable most likely to move the timeline.

Where the roadmap will be shown next

For readers tracking the transition from thesis to silicon, the next scheduled checkpoint is a trade floor rather than an earnings call. Macronix will showcase its latest nonvolatile memory technologies and application solutions at electronica 2026, held November 10–13 in Munich, Germany, with automotive electronics and edge AI among the applications on display at Hall C5, Stand 279.