The chatbot still remains the most popular AI assistant worldwide with over 1.1 billion monthly users, followed by Gemini with 662 million and Claude with 245 million.
On what hardware? I have a 1yo laptop running Linux and it’s slow AF on open source llms. Might be to do with video card drivers, but I’d be happy to hear suggestions on which machine to put next to my raspberry pi for a local LLM.
You don’t really want to be running them on a laptop. You either need a big dedicate graphics card (with 16 GB of RAM preferably) or a unified RAM/VRAM machine like a mac mini or a strix halo.
Local model running is definitely more doable now that it was a year ago but its not yet at the level of throw it on anything and it’ll work fine.
Especially with open weights models available from cloud AI providers.
Yeah the open weight models are getting good enough that a lot of people can just host their own for most tasks.
On what hardware? I have a 1yo laptop running Linux and it’s slow AF on open source llms. Might be to do with video card drivers, but I’d be happy to hear suggestions on which machine to put next to my raspberry pi for a local LLM.
You don’t really want to be running them on a laptop. You either need a big dedicate graphics card (with 16 GB of RAM preferably) or a unified RAM/VRAM machine like a mac mini or a strix halo.
Local model running is definitely more doable now that it was a year ago but its not yet at the level of throw it on anything and it’ll work fine.
I’m talking more enterprises running specialized hardware, but if you needed to do it on a laptop, your best option would probably be strix-halo.