As specialist cloud operators skilled at running high-powered GPUs and other AI infrastructure are emerging, some of the other players are expected to move to AMD instead. It is believed that CoreWeave, Lambda, or Voltage Park, have built their clusters using tens of thousands of Nvidia GPUs,.
Understanding TensorWave
Experts believe that earlier this month, TensorWave, a startup has began racking up systems powered by AMD’s Instinct MI300X. TensorWave is expected to plan to lease the chips at about half of the cost charged to access Nvidia accelerators.
Reportedly the MI300X was launched at AMD’s Advancing AI event in December. The 750W chip is believed to use a combination of advanced packaging that combines 12 chiplets. Additionally it also includes about 20 HBM3 modules, all into a single GPU that is believed to be 32% faster than Nvidia’s H100. Experts believe that the new chip has the potential to have a 192GB of HBM3 memory capable of delivering about 5.3TB/s of bandwidth compared with the 80GB and 3.35TB/s.
What’s next?
From what it is understood, Nvidia’s H200, a version of the H100 having HBM3e memory bandwidth is a major contributor to AI performance, especially in inferencing on large language models. The standard configurations of AMD’s latest GPU require eight accelerators per node, as compared to Nvidia’s HGX and Intel’s OAM designs. “We have hundreds going in now and thousands going in the months to come,” Tatarchuk explained, adding that this is what TensorWave were busy racking and stacking.
Furthermore, TensorWave plans to launch its MI300X nodes using RDMA over Converged Ethernet (RoCE). These bare metal system is expected to be available for specific lease periods. It is believed that the new product will be priced at about $1/hr/GPU.
Follow FE Tech Bytes on Twitter, Instagram, LinkedIn, Facebook
