NVIDIA-acquired Groq moves next-gen LPUs from SRAM to 3D stacked DRAM

HardwareNew21 October 20261 min read

In brief

According to Wccftech, NVIDIA-acquired Groq plans to use 3D stacked DRAM instead of SRAM in its next-generation LPUs. The company is betting on capacity over speed to address the memory demands of growing models. Current Groq LPUs pack 500 MB of SRAM with 150 TB/s of bandwidth.

Next-gen LPUs planned to use 3D stacked DRAM instead of SRAM
Priority shifts from speed toward memory capacity
Current LPUs pack 500 MB of SRAM with 150 TB/s bandwidth
Groq LPX racks are part of the Vera Rubin ecosystem and entered full production in August

Capacity versus speed

According to Wccftech, Groq, which NVIDIA has acquired, plans to move from SRAM to 3D stacked DRAM in its next-generation LPU (language processing unit) chips. The choice signals a shift toward capacity rather than raw speed in inference hardware.

What the current generation offers

Groq's current LPUs carry 500 MB of SRAM and deliver a very high 150 TB/s of bandwidth. SRAM is fast but expensive and limited in capacity; DRAM offers higher capacity at lower cost but trails on latency and bandwidth.

Why now

As AI models grow, memory capacity becomes a bottleneck. 3D stacked DRAM aims to stack memory layers vertically to fit more capacity into the same footprint. Per the report, Groq is tilting the balance toward capacity over speed.

Ties to the NVIDIA ecosystem

The report notes that NVIDIA's Groq LPX racks, announced this year, are part of the Vera Rubin ecosystem and entered full production in August. Paired with NVL72 servers, they are said to reach 3,400 tokens per second.

Not yet confirmed

The information comes from Wccftech's report; neither Groq nor NVIDIA has issued an official announcement or product timeline. It is therefore unclear when the next-gen LPUs will arrive and which manufacturing partner will handle the 3D stacked DRAM.

Why it matters

A useful signal for readers tracking where inference hardware memory architecture is heading, though it should be noted the report lacks official confirmation.

Sources

WccftechFirst reported by
NVIDIA-Acquired Groq Is Moving Its Next-Gen LPUs From SRAM To 3D Stacked DRAM, Betting On Capacity Over Speed As Models Keep Growing

NVIDIA #Groq #Nvidia #memory #inference hardware #DRAM