What simply occurred? At its first large investor occasion since breaking off from Western Digital, SanDisk unveiled one thing it has been cooking as much as take a chunk out of the new AI market. The corporate has a brand new reminiscence structure referred to as high-bandwidth flash that fuses the large storage capability of 3D NAND with the sort of bandwidth provided by HBM.
This hybrid creation stacks up a whopping 16 layers of SanDisk’s newest 3D NAND dies utilizing tiny knowledge pipelines referred to as through-silicon vias. There’s additionally a particular logic layer that may zip knowledge out and in of the person NAND sub-arrays in parallel. This ends in HBF packing 8 to 16 instances extra capability per stack than in the present day’s HBM implementations.
In certainly one of SanDisk’s examples, a system rocking eight HBF stacks might present a monstrous 4 terabytes of capability for storing cumbersome AI fashions like GPT-4 straight on GPU {hardware}.
The important thing innovation appears to be that HBF’s structure breaks with conventional NAND designs by splitting every die into many tiny sub-arrays that may be randomly accessed in parallel, moderately than handled as bigger block-based planes and pages. This allows the excessive bandwidth whereas sustaining NAND’s price and capability benefits.
In fact, NAND’s Achilles heel has at all times been greater latency in comparison with DRAM applied sciences resembling HBM. As identified by Tom’s {Hardware}, HBF is not any completely different and clearly would not match DRAM’s blazing speeds. The brand new structure is being focused at read-intensive AI inference workloads that want excessive bandwidth and capability however can tolerate greater latency. Duties like gaming are off the desk.
There are nonetheless some hurdles for the know-how to clear. SanDisk was quiet on the way it will get across the write endurance limits of NAND, in addition to the difficult block-based addressing that would hamper randomized entry. We additionally do not know what sort of bandwidth numbers HBF can hit.
Regardless of the remaining questions, it appears SanDisk sees large potential. The corporate desires HBF to be an open commonplace complementing HBM in {hardware} like GPUs. To realize this, it is already lining up companions and plans three full generations of HBF improvement, suggesting a critical long-term funding.
Finally, SanDisk even sees the tech filtering down from high-end AI techniques to shopper units like smartphones.
