HBM–HBF Memory Systems for LLM Decoding

Vertically integrated heterogeneous memory systems and near-memory computing for LLM decoding

Status: Manuscript under review.

My ongoing research explores vertically integrated high-bandwidth memory (HBM) and high-bandwidth flash (HBF) systems for near-memory LLM decoding. The work examines how complementary memory technologies can balance bandwidth, capacity, and data movement for model weights and key-value caches.

The project combines cycle-level simulation with analytical modeling to explore the design space of heterogeneous memory stacks and base-die compute. It considers memory allocation, thermal behavior, and cost together when evaluating system tradeoffs.