Biden administration bars China from buying HBM chips critical for AI accelerators

Updated The Biden administration has announced restrictions limiting the export of memory critical to the production of AI accelerators and banning sales to more than a hundred entities.

The trade restrictions, updated [PDF] by the Bureau of Industry and Security (BIS) on Monday, place limits on the sale of high-bandwidth memory (HBM) to countries of concern without a license. In this case, the country in question is the People's Republic of China.

The most sophisticated HBM modules are produced by a handful of vendors – including Korea's Samsung and SK hynix, and US-based Micron. Critically, HBM is an essential component in the high-end GPUs and accelerators used in AI training, inferencing, and scientific computing.

HBM's role in those workloads is spelled out in its name: it offers substantially higher bandwidth compared to conventional DDR or GDDR memory, albeit at higher cost and power consumption.

Memory bandwidth remains one of the biggest bottlenecks for AI and supercomputing performance, so many chip houses have begun prioritizing it over floating point performance in the latest generations of their hardware. Nvidia's H200 and AMD's MI325X are both bandwidth-boosted versions of their predecessors that replace HBM3 with faster HBM3e memory.

The result is particularly noticeable when running the large language models (LLMs) that power popular chatbots like ChatGPT or Baidu's Ernie. The greater the bandwidth, the faster a chatbot can produce out a response – and by extension the more users it can serve.

Under the new rules, HBM producers will need to obtain special export licenses to sell the parts to Chinese firms. Along with the restrictions on HBM, the Biden administration is adding 140 Chinese firms to the US Entities blacklist.

Note that HBM typically makes use of advanced packaging technologies like TSMC's CoWoS – access to which is already restricted for many prominent Chinese chipmakers and tech giants, including Huawei.

China's semiconductor industry is striving to develop products comparable to those the US has effectively banned. As we've previously reported, Semiconductor Manufacturing International Corporation (SMIC) is already working to ramp production of a 7nm process node which has seen use in some Huawei handsets.

Meanwhile, Chinese memory vendors are working on their own HBM. Earlier this year, ChangXin Memory Technologies, aka CXMT, reportedly began setting up testing and manufacturing equipment capable of producing the chips in volume. When the first HBM modules will come off CXMT's assembly line – and what kind of performance they can achieve – remains to be seen.

HBM isn't strictly necessary to support AI applications. Many Nvidia and AMD GPUs still use GDDR memory and can achieve adequate memory bandwidth of 800-960GB/sec. That's far slower than modern HBM, but more than serviceable for inferencing on smaller LLMs like Meta's Llama 8B or Alibaba's Qwen 2.5 7B.

If that won't work, SRAM and scale have also proven effective alternatives to HBM, as demonstrated by the likes of Cerebras and Groq. By allocating large quantities of SRAM to each chip and using high speed interconnects or wafer scale packing to connect them, both developers have been able to achieve extremely high speed throughput for AI inference – even compared to rigs that use standalone HBM. Whether or not SMIC possesses the technology or expertise to replicate these products domestically is another matter entirely.

So, while restrictions to HBM exports to China may be a setback, it won't mean China's AI and semiconductor ambitions become unachievable.

• Submarine cable resilience board announced on same day maybe-cut-by-China Baltic cable repaired
• China launches first next-gen Long March 12 rocket, christens private spaceport
• China has utterly pwned 'thousands and thousands' of devices at US telcos
• China sends cloud powered by homebrew Loongson CPUs into space

The Biden administration has enacted many technology export restrictions to deprive China of technologies related to semiconductor manufacturing and AI accelerators. These efforts have included limiting the export of high-performance chips, and a ban on the sale of extreme ultraviolet and deep ultraviolet lithography equipment required to produce them.

However, recent developments have cast doubt on the effectiveness of these controls.

"We are reviewing the recently updated controls from the US government to understand how they may apply to Micron," a Micron spokesperson told us. "Micron complies with all applicable laws and regulations, including all export control regulations, in every country where we operate." ®

Read next: US curbs HBM exports to China? That's more for the rest of us

Editor's note: This story was amended post-publication with comment from Micron.