Exploring High-Bandwidth Flash (HBF) for Modern LLM-Serving Systems

This talk discusses the potential benefits and technical challenges of an emerging technology called High-Bandwidth Flash (HBF), as a scalable complement to DRAM-based HBM for overcoming the memory-capacity wall in modern LLM-serving. The rapid scaling of LLMs is creating unprecedented demand for memory capacity and bandwidth that conventional HBM alone cannot meet efficiently. NAND flash memory is gaining attention as a promising solution for extending GPU memory at a lower cost. For example, SanDisk first proposed the concept of HBF in 2025, targeting ~20x larger capacity with matching bandwidth compared to DRAM-based HBM, and major memory manufacturers are now actively exploring its realization. In this talk, I will present our ongoing research to systematically explore the potential be