Where is China’s Groq?
In late 2025, HBM was all the rage. You need HBM for AI inference; inference decode is a memory-constrained operation, and only HBM was able to reach bandwidth speeds to serve models effectively.
But the market has matured since then, and there is no longer one architecture to rule them all. Different silicon works better for training, prefill, or decode. The HBM shortage has created desperation too, opening lanes for new chip design startups to thrive. These companies, including Groq, Cerebras, Positron, Tenstorrent, d-Matrix, MajesticAI, and others, are formed on the thesis that you don’t need HBM for inference decode. Instead, you can design chips cleverly to reduce the need for supply-constrained and low-storage HBM by utilizing cheaper and more plentiful SRAM or DDR.
The world realized this at the latest in mid-2026. China has been living in this situation for much longer but has only recently started up its HBM-free chip efforts. HBM has been export-controlled since 2024, and Chinese companies certainly have been attempting to develop better chips to train and serve Chinese models. Their capital ecosystem reflects this push: in 2020 alone Chinese VC funding threw $13 billion at chip design firms, and their funding for chip startups has dwarfed that of American VCs every year since at least 2016.¹
So what gives? If the answer isn’t money, why is Silicon Valley seeming to pursue HBM-alternative architectures more vigorously than China?
We cover:
Why the problem is fab space
The difficulties with mixing different chips with Chinese model architectures
The possibility that it’s an America problem, not a China one
And what’s on the horizon for China’s chip industry
Limited Fab Space
The binding constraint on China’s ability to experiment with alternative chip architectures is limited advanced-node (7 nm and below) fab space. Regardless of architecture, more advanced nodes beget better chips, and plenty of companies are fighting to get a slice of node capacity.
Designing the chip requires negligible fab space. If SMIC operates anything like TSMC or Global Foundries, many chip prototypes can be split across the same wafer, and iterating through multiple prototypes consumes almost zero resources for a fab spitting out tens of thousands of wafers per month.
The problem for design startups isn’t fab space to prototype but rather fab space to manufacture after tapeout. SMIC’s advanced node capacity is estimated at 60,000 wafers per month, and Huawei gets first dibs on that fab space. Their Ascend-line AI chips are the primary choice for domestic models attempting to migrate from Nvidia; Z.ai’s GLM-Image and Meituan’s LongCat-2.0 were trained on Ascends, and DeepSeek’s V4-Pro was post-trained on the chips.
Even including the Ascend line, the largest share of SMIC’s advanced node capacity is used up by Huawei’s smartphone chips. Their Kirin line was the original use case for SMIC’s 7 nm node, and given that smartphones are a commodity product, these chips take up more fab space and orders compared to any AI accelerator or GPU. The rest of SMIC’s fab space is fought over by companies like Alibaba, Baidu, Tencent, and smaller design houses.
Cambricon, which manufactures its Siyuan 690 on SMIC’s 7 nm, and similar Chinese companies are the exception, not the rule. Cambricon entered the AI business in 2018, far before the space was crowded and larger companies were designing their own silicon. It is hard to imagine a new Cambricon being founded today to design and make a non-HBM chip for the Chinese market today. The risks are too high for investors, and other players already crowd the fab space. This helps answer why China seems to lack a Groq, Cerebras, Positron, or MajesticAI. The ecosystem doesn’t reward no-names to come in and succeed.
If founders hope to make such companies, they cannot turn to domestic nodes. Instead, like many of China’s AI accelerators, they would have to resort to manufacturing more mature-node chips with TSMC, which could result in a noncompetitive product. Groq’s first-generation LPU was built on a 14 nm node, meaning such products aren’t out of the question, but even Groq is now migrating to 4 nm.
Insufficient Demand
The second structural reason for the absence of these sorts of chips is the lack of inference demand up until recently. China has been trailing the U.S. in AI adoption for as long as the AI race has existed, and the token spends prove it. Chinese companies just do not sell as many tokens as American companies do.
All AI usage has increased exponentially over time, though China’s national usage is only barely beating the usage of Google’s AI models. OpenAI and Anthropic don’t report their tokens; adding the two to the count for ex-China token usage would show that China is years behind in token spend.
While the U.S. may have reached the point where inference demand was high enough to warrant its own chips years ago, Chinese demand had not reached that point until very recently. Whereas Western chips serve the entire world ex-China, Chinese chip design startups can only justify their existence if the Chinese domestic market has sufficient demand. If you are running DeepSeek in the U.S., it will be run on a world-class Nvidia chip built on the most advanced node with high-end HBM.² Only when you are running a model in China will you have the need for a different kind of chip.
Open-sourcing models also reduces the need for serving inference. Users download the weights and run the model on their own compute, meaning model developers don’t have to optimize for inference themselves. This has the added benefit of utilizing what limited compute they have nearly entirely for training.
In this light, it makes sense that Chinese firms haven’t been funding inference chips. How can they design and manufacture an inference chip on a worse SMIC node that could compete with Nvidia and AI accelerators built on better nodes and with access to better memory? The market must incubate until domestic demand is sufficient.
Now, however, domestic demand is appearing to become sufficient. The combination of coding agents, better models, and time has led Chinese companies to think that it’s time to design their own silicon. Z.ai, DeepSeek, and ByteDance are all intending to create their own inference chips. Only ByteDance is confirmed to be pursuing a non-HBM architecture, though our money is on Z.ai and DeepSeek following suit.
China’s Menu of Architectures
Non-HBM chip architectures also have not been pursued for another reason: they often present various manufacturing difficulties and costs, and perform worse compared to HBM architectures. As China’s demand for alternate architectures picks up, it’s worth looking at what is on the menu of possibilities for Chinese chip designers and weighing their benefits and drawbacks. So far, the three main options are DDR-based architectures (d-Matrix, MajesticAI, Tenstorrent), SRAM-based architectures (SRAM, Groq), and ByteDance’s wildcard bet on ReRAM.
DDR Architectures
DDR (double data rate, the most common type of commodity RAM memory) is set to be China’s most important HBM-alternative bet, with the majority of Huawei’s chips this year set to be the DDR-based 950PR. The difference between DDR-based chips is entirely around how it is being packaged. The simplest solution places DDR chips, instead of HBM, alongside the XPU on the interposer. This is effectively ‘budget-HBM’ where you sacrifice the additional capacity and bandwidth that HBM would bring you, but get significantly lower cost and still reasonable bandwidth. There are then more ambitious strategies that look to use hybrid bonding to place the DDR directly on top of the XPU, or which use system design to compensate for DDR’s weaknesses.
Huawei is going for a simple design for its DDR chips. Due to CXMT’s struggles in scaling its HBM production and exhausted pre-export-control HBM stockpiles, its 950PR instead uses DDR to attain decent performance at relatively low cost. The challenge is that while saving costs on HBM, it generates much higher system costs. To compensate for the lower memory capacity, you need large optical scale-outs of larger numbers of chips, meaning higher power costs and lots of expensive transceivers. To match a single B300 — 5 PFLOP of dense FP8 and 8 TB/s of bandwidth — needs roughly five Ascend 950PRs³, which together carry twenty optical modules, compared to just three for the B300’s scale-out domain, with it not needing optics at all for its 72-GPU copper scale-up domain.
All these optics are a price Huawei is willing to pay. It needs to supply the domestic compute ecosystem and has existing strength in networking. Combined with an ability to own and optimize the end-to-end optics system, this sets it up for strong performance and cost reduction in its optics fabrics relative to Western hyperscalers.
It isn’t clear from Huawei’s published paper on the 950’s architecture exactly how the 950PR integrates the DDR memory. The diagram above encompasses both the 950PR and the variant that uses HBM, the 950DT, but simply indicates ‘Global Memory,’ which could cover either the 950PR’s DDR solution or the 950DT’s HBM solution, giving little insight into either. The white paper notably doesn’t mention HBM, instead referring to “high-speed on-chip memory” 高速片上内存. Huawei could be using a non-standard HBM variant, or simply be uncertain about its HBM production ramp, and so obfuscating in the white paper — it is hard to say.
We do know that the 950PR was intended to use Huawei’s HiBL 1.0 memory, though this term is dropped from the white paper. The 950PR has eight memory modules, delivering 128 or 112 GB of capacity, at 1.6 or 1.4 TB/s of bandwidth. That is under half of the B300’s 288GB and a fifth of its 8 TB/s bandwidth. Most likely the 950PR’s memory is LPDDR modules, with a custom memory interface that sits on the wider substrate package, but they could also be custom wide-bandwidth DRAM or a kind of HBM-lite with smaller stack heights. The first seems the simplest and likely most economical. Illustration of the various setups by Claude.
The 950DT and 960DT both deliver significantly higher bandwidth, commensurate with effectively an HBM3E solution. The 950DT production timeline sits behind the 950PR almost certainly due to the challenges of scaling HBM fabrication at CXMT or other Huawei-affiliated fabs. That the 960DT was brought forward three quarters from Q4 2027 to Q1 2027, suggests that Huawei’s chip and system design is running ahead of the HBM supply, and so priority is being given to getting those scarce HBM dies into chips with superior performance… leaving the 950DT potentially a dud generation, set for only fairly small-scale production.
The 950PR and the 950DT were designed to be complementary, with the ‘PR’ standing for ‘prefill and recommendation’, which are compute-bound and so don’t require HBM-like memory, and the ‘DT’ standing for ‘decode and training’, which do require high bandwidth and therefore HBM. The open question is whether Huawei is shipping any integrated systems of the two, given the production constraints on the DT. It seems possible that pure 950PR systems could be shipped for the time being until the DT scales up later this year.
The more ambitious version of DDR chips uses hybrid bonding. Rather than being packaged akin to HBM, where the memory sits alongside the logic on a silicon interposer, with bandwidth limited by the shoreline of the logic processor, the DDR is hybrid-bonded on top of the logic. By vertically stacking the DDR on top of the processor, you shrink the communication distance and dramatically raise potential bandwidth by opening up far more possible interconnects. However, hybrid bonding is difficult, expensive, and brings many novel challenges such as thermal management.
The Shanghai-based start-up Oriental Computing 东方算芯 recently revealed such a hybrid-bonded, memory-on-logic chip, the DF1000. It claims an impressive 6.4 TB/s bandwidth, using only a 14nm logic node and 18nm DRAM, that is, bandwidth about 80% as high as the Nvidia B300’s 8 TB/s. Oriental Computing also has an aggressive roadmap, claiming that its DF2000 will hit 15 TB/s and ship by the end of this year. Moreover, it will integrate into a TY64 SuperNode, which would offer almost twice as much memory bandwidth in total as a GB200 NVL72 system.
Oriental Computing was founded by Wei Shaojun (魏少军), a Tsinghua professor and prominent figure in China’s semiconductor industry. However, all of the specs above are simply claimed by the company itself with no third-party verification. There is no public information on its customers or commercial uptake. Hybrid bonding remains a technically nascent approach, so it remains too early to properly assess Oriental Computing’s roadmap.
The final contender in the hybrid-bonded DRAM camp is Xiaomi. Though it is not yet producing AI server chips, it is producing its Xring O100, which is designed for edge inference of models within devices. By hybrid bonding two custom DRAM wafers onto the logic, it allows for much faster bandwidth than traditional on-device chips. However, the memory capacity remains very small, suitable only for tiny models running on the edge. One advantage that this small scale brings is that Xiaomi can, and reportedly is, producing these chips at TSMC, taking advantage of their 6nm node and wafer-to-wafer hybrid bonding capabilities. This is because restrictions on TSMC producing AI chips or Chinese customers apply only to chips designed and marketed for use in data centers, not small edge-focused chips like the O100.
The last theoretical DDR architecture is based on optimizing chip interconnects rather than memory bandwidth, similar to Tenstorrent’s thesis. No Chinese firm currently seems to be pursuing this path, but it is an option on the menu for future endeavors. This path is best enumerated in a by user Big Boss, which received the badge of honor reply “Interesting” by Elon. Big Boss points out that because of model sharding in modern inference, the limiting factor in inference decode is not memory bandwidth but rather inter-chip latency.
Big Boss’s architecture uses LPDDR instead of HBM, as they take approximately the same amount of time to access, and compute is minimized to match the provisioned memory bandwidth rather than leaving unused FLOPS during decode as modern GPUs do. Instead of wasting space on increased FLOPS, the chip can dedicate more space to ports for a switchless interconnect mesh, thus optimizing interconnect latency. His theoretical architecture would work well with just a six-nanometer node, shedding the necessity of cutting-edge nodes. Such a chip would also not require advanced packaging, as we’ve cut out the HBM.
We’re not entirely sure how commercializable this architecture is, but it is interesting.⁴ It’s important to note this architecture seems to optimize speed of tokens to small batches of users rather than maximal throughput, though Big Boss argues his architecture is on par with Nvidia racks on throughput. For long-context applications too, this architecture may break down as maximum bandwidth actually becomes a need. Further, it’s unclear who in China would want to adopt this framework. The chip would call for short, electrical, and switch-free interconnects, whereas Huawei is known as an optical interconnect company; optical interconnects slow token speed, as the electro-optical conversions, switch hops, and other factors add time to the process.
SRAM Architectures
The poster child for SRAM architectures is Cerebras, whose Wafer Scale Engine (WSE) takes up an entire silicon wafer to function as a singular chip. Cerebras’s architecture makes a trade-off: higher-bandwidth and in-supply SRAM instead of higher capacity and cheaper HBM.
The problem is that Chinese models were built with compute and memory scarcity in mind. Since the DeepSeek moment in January 2025, Chinese models have achieved token efficiency by taking MoE sparsity to its limit. By minimizing the number of parameters activated for each token, Chinese models can reap the speed and cost benefits of a smaller model while obtaining the performance of a large one.
However, this sparsity assumes that parameter storage is abundant, which is the case for DRAM but is not for SRAM. SRAM storage real estate is expensive, and companies must pay for the whole plot even though only minuscule portions are being used at any given time. The lightning-quick SRAM bandwidth helps with token speed, but it doesn’t reduce the cost or the need to store all the weights.
Especially as models get larger, these chip architectures are less and less suitable for Chinese models. To accommodate large model weights, multiple Cerebras wafers must be strung together, and one wafer alone costs $2 to $3 million (though we’ve heard rumors that the price is now down to the sub-million). Further, by needing to string multiple wafers together, the speed gains are canceled by the slower communication of data between wafers. The interconnect speed on WSE-3 is 1.2 Tb/s per wafer, a steep decline from the on-wafer bandwidth of 21 PB/s. The WSE-4 doubles the interconnect speed, but it’s still a far cry from the on-wafer bandwidth.
For example, DeepSeek-V4-Pro has 1.6T parameters at mixed FP8/FP4 precision. With each Cerebras WSE-3 containing 44 GB of SRAM storage, at least twenty wafers would be required just to hold the model weights at native precision. In reality, this number would be over seventy for Cerebras’s current architecture, which can only handle FP16. Even if China could create a similar architecture that accommodates FP4, the number of wafers would still be larger due to the need to store KV cache on the same wafer.
This implies that Cerebras-like chip architectures are most cost-effective for smaller, denser models. The problem is that the industry, and particularly the Chinese AI industry, continues to move toward larger, sparser models. The industry is trending toward valuing cheaper storage, whereas Cerebras offers more expensive storage.
Using this sort of chip architecture also negates the niche that Chinese models have dug out for themselves. Chinese models dominate markets where users want cheaper tokens and are willing to use slightly worse models. Cerebras’s thesis is to offer more expensive tokens at greater speed. If a user is already willing to pay Cerebras prices for speed, why would they use a worse model underneath? You wouldn’t buy premium fuel for a cheap car.
Taken all together, SRAM-based architectures do not suit Chinese MoE models. The large parameter counts require expensive SRAM storage, and Chinese models have hitherto been destined for the cost-sensitive market.
ReRAM Architectures
The biggest wildcard architecture is resistive random-access memory (ReRAM). ByteDance is currently pursuing a ReRAM-based AI accelerator, with the ReRAM coming from InnoStar (昕原半导体), another Chinese company.⁵
ReRAM was invented in the 1960s and has been floating around as a serious prospect since at least the early 2000s, when everyone incorrectly thought NAND scaling was coming to an end. With ReRAM, information gets stored as electrical resistance — not electrons like DRAM or NAND; it manipulates atoms within a conductive filament, and the changes in that filament’s conductivity are the data.
ReRAM has a lot going for it. It does not require leading-edge lithography, with most ReRAM products hovering around 28/22 nm nodes, and ReRAM is even denser than SRAM. By using ReRAM for compute-in-memory or compute-near-memory, chipmakers can circumvent the bandwidth problem. In particular, ReRAM is conducive to storing model weights, which rarely change yet take up increasingly more storage. Although writing is expensive and slow for ReRAM, this matters less for inference, where the model is ideally written once and then only read.
However, ReRAM’s lack of adoption is not because the rest of the industry overlooked it. Its limitations are myriad, and ByteDance will have to solve many open problems before the technology can be commercialized for AI inference. ReRAM has a retention problem for mass storage: it loses its stored value within seconds at low programming currents; the need for a higher programming current cancels the density and energy efficiency gains ReRAM originally had compared to SRAM.
Drift is also a problem. Filaments do not perfectly preserve the conductance they were set to, but gradually drift beyond an acceptable range. Some studies have found ways to account for this error, but these methods likely confine the chip to an individual model. This may be worth the cost to ByteDance, but it’s unclear how tenable this is as models update.
Lastly, the research on running LLMs on ReRAM is embarrassingly thin.⁶Most studies test ReRAM for image classification models, but only a handful of studies apply the technology toward LLMs. Those that do use simulations rather than real chips, and the one study we found that did use a real chip did not specify how large the model was. The largest parameter count we could discover was less than four million, which pales in comparison to billion-parameter class “small” models, let alone frontier ones; one study simulates inference on Llama2-7B, but it maps only the model’s linear layers on ReRAM, while attention and KV Cache stay off-ReRAM.
It is unclear how ByteDance will resolve these issues with ReRAM, but perhaps it will become the path of least resistance as HBM remains blocked off.⁷ Perhaps ReRAM’s niche will be to serve image classification and recommendation systems, freeing up GPUs and other accelerators for LLM inference.
We’re interested to see what ByteDance does with this technology, and if anyone has any leads, thoughts, or corrections, please reach out to [email protected].
China’s Upcoming Chips
Despite all that is working against Chinese sans-HBM chips, China’s leading companies may be finally taking a stab at some alternatives. DeepSeek and Z.ai are pursuing their own inference chips, though it remains to be seen whether their silicon will require HBM. ByteDance’s ReRAM is being developed by another Chinese company, Innostar.
In the near term, the landscape is dominated by firms building HBM-based chips: Cambricon, Alibaba, Baidu, Biren, MetaX, Enflame, etc. Only Huawei is taking a large-scale non-HBM bet in the next year or two, with its 950 and then 960PR looking to use DDR to circumvent HBM bottlenecks and try to achieve large-scale production of usable chips. Longer term, ByteDance’s bet on ReRAM and Orient Silicon’s bet on hybrid bonding are the most prominent HBM alternatives to watch.
For any of these chips to succeed there needs to be a combination of:
Competitive design
Available manufacturing capacity
A sufficient software stack
Willing buyers and partners
Huawei is best positioned because it achieves (to at least some degree) all four. Internal chip efforts from hyperscalers or AI labs can at least guarantee willing buyers, and have a likely edge in software development. For an effort such as Orient Silicon’s you don’t necessarily have any of the four, with design chops likely but not yet proven, hybrid bonding capacity a big question mark, and it is not clear if there is a sufficient ecosystem of software developers and partners.
Limitations imposed by export controls will make Chinese firms more creative, and potentially push them to explore alternative architectures earlier than their chip design peers outside China. The performance boosts these can give are useful but cannot fully compensate for the overall performance deficit caused by advanced node manufacturing weakness. Innovations such as hybrid-bonded memory on logic are equally on published roadmaps at Nvidia and elsewhere.⁸ China’s capabilities at the chip level will continue to lag until it can crack extreme ultraviolet lithography or alternative techniques that enable real transistor shrink to more advanced nodes.
The strongest path for China’s compute efforts will remain Huawei’s strategy, which entails accepting weaker, less efficient individual chips and focusing on optimizing at the system level. At a wider level, it means making use of China’s greater ability to make use of large, energy-inefficient clusters, and working to simply produce as many SuperPods as possible given the manufacturing constraints in place. This is more of an industrial than design challenge, ill-suited to lean chip design startups, and instead calling for a more expansive firm such as Huawei which can become the central coordinating node within this wider industrial system.
China does not ultimately need its own Groq; it needs its own ASML, TSMC, SK Hynix etc., the whole complex industrial ecosystem on which Western chip design startups can thrive. Interesting novel chip startups are worth tracking, but this industrial-scale indigenization effort is the real story for Chinese compute.
This tallying of funds has some limitations, though. Design houses, which are most of the startups getting funded by the kind of capital discussed above, aren’t the only ones designing chips and likely aren’t even the most important designers. Larger conglomerates like Huawei, ByteDance, and Alibaba are not counted in these figures, nor are semiconductor integrated device manufacturers, nor are AI labs that have begun to design their own custom silicon! In this sense, American companies as a whole may have more funding to design chips; OpenAI’s recent Jalapeño chip, for example, would be missed by the above figures. Still, the point remains that if a Chinese company wanted to design an inference chip without HBM, they would not be in want of funding. Funding an IC design house is significantly cheaper than funding a fab, and China does not have any reservations about the latter. Designing silicon requires no real capex, since costs are usually in talent, software, IP, mask sets, and other relatively low-ticket items.
This is true for most users running DeepSeek locally or through a third-party inference provider. This is not true for those accessing the DeepSeek API through their website, in which case inference would be run on Chinese servers.
Huawei rates the 950PR at roughly 1 PFLOP FP8 and 1.6 TB/s of memory bandwidth. LightCounting estimates 4,096 800G optical modules across a 1,024-NPU Atlas 950 configuration—four per NPU on average, counting both compute- and switch-side optics. Applying that ratio to the 950PR assumes it uses the same SuperPoD network topology.
The main difficulty, as we have been told, is in managing latency.
ByteDance holds a significant stake in Innostar, alongside Ant Group.
Other potential problems exist, including the right choice for electrodes and dielectric, write endurance, read speed, and the lack of a proper software ecosystem.
For what it’s worth, experts we spoke to do not want to touch ReRAM with a ten foot pole, but it is interesting that ByteDance is still pursuing it.
See this piece from Epoch for a much more detailed comparison between Huawei and Nvidia’s technical roadmaps
You’re on the free list for ChinaTalk. Please consider supporting the newsletter and unlocking exclusive paywalled content by becoming a paying subscriber.
© 2026 Jordan Schneider
.