inari@piefed.zip to Technology@lemmy.worldEnglish · edit-27 hours agoDeepSeek ditches Nvidia for Huawei chips in V4 launchcybernews.comexternal-linkmessage-square24fedilinkarrow-up1125arrow-down15
arrow-up1120arrow-down1external-linkDeepSeek ditches Nvidia for Huawei chips in V4 launchcybernews.cominari@piefed.zip to Technology@lemmy.worldEnglish · edit-27 hours agomessage-square24fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up4arrow-down1·edit-26 hours agoJust not power/cost efficiently on CPU only, is what I meant. CPUs don’t have the compute for batching (running generation requests in parallel). You need an accelerator, like Huawei’s, to be economical. It’s fine for local inference, of course.
minus-squareag10n@lemmy.worldlinkfedilinkEnglisharrow-up1·4 hours agoA whole ecosystem that can run on any hardware, efficiently or not, is a whole ecosystem developed for the Chinese market
Just not power/cost efficiently on CPU only, is what I meant. CPUs don’t have the compute for batching (running generation requests in parallel). You need an accelerator, like Huawei’s, to be economical.
It’s fine for local inference, of course.
A whole ecosystem that can run on any hardware, efficiently or not, is a whole ecosystem developed for the Chinese market