Chinese companies are heavily incentivized to use Chinese chips instead of American since Trump blocked trade with China.
China used to parallel import the chips they needed, and even repackage them with more onboard RAM, making more powerful Nvidia solutions available in China than in the rest of the world.
But Trumps behavior towards China made the Chinese government decide to limit the use of American technologies for AI.
There was a point where Nvidia exports to China was basically at a standstill, because China forbade the purchase of a new cut down Nvidia chip made for the Chinese market to circumvent American trade restrictions.China is building their own complete stack now, replacing everything with Chinese technologies, right from the AI chips to the entire AI software framework.
So not only does Nvidia and other American companies lose hardware sales, the entire stack will be threatened with a Chinese alternative, that will likely compete with American options on the international market in the future. If Cuda loses its current dominance, it will be easier for competitors to take marketshare from Nvidia.
Hopefully this will be good for consumers worldwide.
Qwen is already the standard for actual pros as far as I can tell.
Hopefully this will be good for consumers worldwide.
Until America decides to tariff anyone using Chinese technology.
It’ll be good for consumers worldwide. America is not the whole world.
I, for example, am in Canada. We’ve established a bunch of very nice trade deals with China recently, we’re going to end up with access to a bunch of Chinese products that Americans can’t get due to their self-imposed trade war with China.
I think they mean that the US would put trade pressure on countries doing any tech trade with China, not specifically preventing it punishing American companies from using Chinese chips.
Unfortunately the United States is still a big economy regardless of their politics and Manny is right that the US would throw their weight behind anti China policies to the detriment of other nations.
How successful such a move would be is up to debate.
The US is already trying to throw its economic weight around bullying Canada, and we’ve already settled in to an effective economic defensive posture. Those trade deals with China are actually part of it, previously we were supporting various American initiatives to tariff China but the Americans tore up a bunch of agreements with us so we responded in kind. It’s unfortunate but they started it and we’re prepared to hold our own.
America decides to tariff anyone using Chinese technology
America already decides to tariff countries regardless of what they’re doing
US tariffs hold now power anymore these days.
What’s left unsaid is the software architecture is extremely interesting, and efficient.
Ironically, the Nvidia embargo was the best thing to ever happen to the Chinese labs (which Nvidia tried to tell the US govt). It forced them to get thrifty, unlike US labs which (allegedly) fill some GPU farms with busywork for the appearance of high utilization.
the only thing deepseek is really good for is to fuck over NVIDIA and chat gpt or American ai in general because deep seek is bast on chat gpt after all anything that brakes this monopoly is good and anything associated with it like ram.
You can run it on CPU alone. Not surprising they’re building their own AI ecosystem
I mean, sure. You could also run it by drawing marks in sand. It doesn’t make any sense to do either, though.
It’s still matrix multiplication. Running it on a general purpose CPU is inefficient.
Not at scale. Even on the new architecture, one really needs some kind of accelerator to make it economical for servers.
Bitnet-like models might change the calculus, but no major trainer had tried that yet.
Even with a bitnet, it’s almost definitely better to train on a high precision float then refine down to bits.
I would expect bitnet to require more layers for equivalent quality too.
I just meant for mass inference serving.
Yeah, I haven’t seen much in the way of bitnet training savings yet, like regular old QAT. It does appear that Deepseek is finetuning their MoEs in a 4-bit format now, though.
Yes, you can run it at scale. Which is why it uses Huawei hardware.
You can run it on anything, scaled or not
Nope! You don’t know what you’re talking about. At all. But you can have fun running a 1.6 trillion parameter model on CPU at basically 0 tokens per second at scale, MoE or not.
Just not power/cost efficiently on CPU only, is what I meant. CPUs don’t have the compute for batching (running generation requests in parallel). You need an accelerator, like Huawei’s, to be economical.
It’s fine for local inference, of course.
A whole ecosystem that can run on any hardware, efficiently or not, is a whole ecosystem developed for the Chinese market
Sad trombone.
