I’m not as up to date (like at all lol) with specifically open source ai. Is it any better then corporate ai? Including data center-like resources use? or otherwise?
For corporate use, it’s very good. But the immediate advantage is cost.
The open source models are sparse, and use an efficient architecture. They are very efficient to run.
You can pick from hundreds of competing providers (including those using specialized ASICs), rent GPUs, or run your own server internally.
I’d say the big unrealized advantage is flexibility. Companies can fine tune the big open models to their specific tasks, and even host a “mixture of Loras” as a set of specialized models. They can constrain output at the sampling level, and cache context programmatically. They can use raw completion syntax creatively, or train task vectors real quick. There’s all sorts of neat hacking to be done, but the issue is that 99.9% of businesses have no idea because they’re used to getting chat response tokens from a black box like Claude.
Now… as a pure, turnkey, “cost is no object” agenic worker? Clause is a but better, most of the time. But not always, these days, especially if speed, long context or other things are factors.
Also… the “unspoken” business issue is that most big open models are from China.
It’s stupid. It’s just a block of weights in an open source harness, no more security risk than (say) a tire on a car. But try explaining that to decision makers who don’t really understand the tech and are afraid of getting hacked.
But Nvidia has a pretty good big MoE now, and there a number of “laundered” finetunes of Chinese models, so even that’s hardly an excuse anymore. It’s just a matter of time before word gets around, businesses realize Claude is never going to replace their employees (just augment them), and look for cheaper and more controlled solutions.
Don’t know exactly how it compares but I’ve measured the electricity increase while running it, something you can run on one graphics card is around the same as a resource intensive video game.
Yes, you can run Qwen 3.6 27b on your laptop and have it do useful work, and it’s more capable than frontier models from a year ago or so. https://qwen.ai/blog?id=qwen3.6-27b
I’m not as up to date (like at all lol) with specifically open source ai. Is it any better then corporate ai? Including data center-like resources use? or otherwise?
For corporate use, it’s very good. But the immediate advantage is cost.
The open source models are sparse, and use an efficient architecture. They are very efficient to run.
You can pick from hundreds of competing providers (including those using specialized ASICs), rent GPUs, or run your own server internally.
I’d say the big unrealized advantage is flexibility. Companies can fine tune the big open models to their specific tasks, and even host a “mixture of Loras” as a set of specialized models. They can constrain output at the sampling level, and cache context programmatically. They can use raw completion syntax creatively, or train task vectors real quick. There’s all sorts of neat hacking to be done, but the issue is that 99.9% of businesses have no idea because they’re used to getting chat response tokens from a black box like Claude.
Now… as a pure, turnkey, “cost is no object” agenic worker? Clause is a but better, most of the time. But not always, these days, especially if speed, long context or other things are factors.
Also… the “unspoken” business issue is that most big open models are from China.
It’s stupid. It’s just a block of weights in an open source harness, no more security risk than (say) a tire on a car. But try explaining that to decision makers who don’t really understand the tech and are afraid of getting hacked.
But Nvidia has a pretty good big MoE now, and there a number of “laundered” finetunes of Chinese models, so even that’s hardly an excuse anymore. It’s just a matter of time before word gets around, businesses realize Claude is never going to replace their employees (just augment them), and look for cheaper and more controlled solutions.
Don’t know exactly how it compares but I’ve measured the electricity increase while running it, something you can run on one graphics card is around the same as a resource intensive video game.
Yes, you can run Qwen 3.6 27b on your laptop and have it do useful work, and it’s more capable than frontier models from a year ago or so. https://qwen.ai/blog?id=qwen3.6-27b