HBM

High-bandwidth memory is a vertical stack of DRAM chips bonded together with through-silicon vias, a technology that lets data move between layers at far higher rates than conventional side-by-side chips allow. A single HBM stack typically contains 8 layers of memory die, each 1.5 mm thick, stacked to a total height of around 12 mm. The throughput of a single HBM stack at current generations reaches 460 gigabytes per second or more, compared to perhaps 50 to 100 GB/s for traditional memory arrangements used in general-purpose CPUs.

The core advantage is not capacity but speed and power efficiency. Graphics processors, AI accelerators, and high-performance computing chips need to move enormous amounts of data per clock cycle. A single GPU may pair with multiple HBM stacks: NVIDIA's H100 uses up to 141 GB/s per stack, with bandwidth distributed across several stacks in parallel. This architecture solves the memory bottleneck that would otherwise limit how much compute work a processor can actually do before it starves waiting for data to arrive.

Why the vertical stack matters

Each layer connects to its neighbors through thousands of tiny conductive holes drilled through the silicon itself. This vertical path is orders of magnitude shorter than the horizontal traces on a circuit board, which means lower latency, less signal degradation, and better use of power per bit moved. The tradeoff is manufacturing complexity: bonding dies with micron-level precision, managing heat dissipation through a tall stack, and testing for defects in every layer before assembly is complete.

HBM adoption began in specialist markets: high-end graphics cards in 2015, and has since expanded to data center GPUs, server accelerators, and some next-generation CPUs. The standard is maintained by JEDEC, with HBM2, HBM2e, and HBM3 variants released at intervals. Each generation roughly doubles bandwidth and capacity per stack while managing thermal constraints that come with stacking more transistors into the same footprint.

Cost and yield remain barriers to broader adoption. HBM is more expensive to manufacture than conventional memory and cannot tolerate the defect rates that DRAM fabs have learned to work around. For consumer equipment or cost-sensitive applications, conventional memory buses still dominate. But wherever raw compute power per watt matters more than bill of materials, HBM has become essential infrastructure.