Groq plans stacked DRAM for its next inference chip to fix a memory problem

The Nvidia-backed chip designer reportedly wants 3D-stacked DRAM to give its LPUs room for trillion-parameter models

Groq built its reputation on speed. Its next chip may finally get the storage space to match.

Industry sources revealed on October 20, 2026, that Groq plans to integrate 3D-stacked DRAM into its upcoming Language Processing Unit inference chips. The goal is to raise memory capacity and bandwidth for large AI models, without giving up the raw throughput that made the company’s hardware stand out.

The plan centers on 3D-stacked DRAM, often shortened to 3DS, connected with through-silicon vias, or TSVs.

According to the research findings, Groq’s version is meant to expand vertical DRAM capacity using standard interfaces. The emphasis is on scalability rather than chasing the most extreme bandwidth numbers possible.

The Groq 3 LPU carries 500 MB of on-chip SRAM. That memory delivers 150 TB/s of bandwidth per chip, which is approximately seven times the HBM bandwidth of comparable GPUs.

Bandwidth is what matters most for token generation, the step where a model spits out words one after another. Groq’s design lets the Groq 3 LPX deliver up to approximately 3,400 tokens per second.

The problem is capacity. A full 256-chip LPX rack offers only 128 GB of SRAM in total.

Daily. Free. Join 34,000+ readers across crypto, finance, and policy.

That sounds like a lot until you meet a trillion-parameter model. Models at that scale force operators to spread the work across many chips, simply to hold the model’s weights in memory.

Stacked DRAM is meant to address exactly this. The research describes the shift as a response to the capacity limits of the current SRAM-centric design, while retaining the high memory bandwidth needed for efficient token generation.

Groq was founded in 2016 by Jonathan Ross, a former designer of Google ’s TPU chips.

In December 2025, Nvidia struck a deal worth approximately $20 billion that covered licensing Groq’s intellectual property and hiring its key personnel. The research findings characterize the arrangement as effectively an acquisition.

Groq also does not appear to be alone. Initial reports suggest that moves toward stacked DRAM, similar to those pursued by Cerebras for its CS-6 wafer-scale chip, are becoming more common as the industry adapts to larger AI models.

The most important signal is philosophical. Groq’s pitch has long been that bandwidth beats everything for inference. A move to stacked DRAM concedes that capacity can no longer be treated as somebody else’s problem.

For Nvidia, the research suggests the effort could strengthen its positioning in processing power for increasingly complex models.

Groq’s reported preference for standard interfaces over ultra-high bandwidth approaches looks like an attempt to keep manufacturing risks manageable.

Things to watch next include any official confirmation from Groq or Nvidia, the capacity figures attached to a new LPU, and whether the 256-chip rack configuration changes. A meaningful jump beyond the current 128 GB per rack would tell you the plan is working.