On 26 August NVIDIA added NVHBM to NVLink Fusion - memory whose base die now carries their own controller. The claim is up to 30 per cent more bandwidth, 15 per cent less power and up to a quarter of the compute die freed up. Amazon's Annapurna Labs goes first.
- The memory controller traditionally lives on the compute die and eats area that could otherwise be doing arithmetic.
- NVHBM moves it into the base die of the HBM stack and hands a ready implementation to several memory vendors.
- The move is aimed at other people's accelerators - NVLink Fusion is the door to semi-custom silicon.
Space on silicon is like space on a stage. Whatever you give to the equipment, you take from the performer.
On every accelerator today, part of the die computes nothing - it serves the memory. The controller sits there because that is how it has been done for years, occupying area that could otherwise be doing arithmetic. On a chip costing tens of thousands of dollars, that is expensive storage space.
Who this is really for
Not their own accelerators. NVLink Fusion is the programme through which other people's chips attach to NVIDIA's network and ecosystem - which is to say, exactly the companies building custom silicon so they depend on NVIDIA less. Amazon is the textbook case.
That is what makes the move more interesting than the numbers. If you build your own chip to get free of one supplier, and the memory, the network and the standard around it are still theirs, the escape is partial. What is being sold is independence running on someone else's rails.
The numbers, of course, are theirs, quoted against standard HBM4E, with no independent verification. Thirty per cent of bandwidth and a quarter of the die freed are serious gains if they hold up in a shipping product - and no such product has been announced yet.
If you follow the accelerator market because of what it costs you, watch this direction. The fight stopped being about whose chip computes faster a while ago; it is about whose memory and whose network sit underneath it.