deleted by creator
Gotta wait till Qwen 4 for proper font scaling.
Ok, but all of them are far too inefficient for what they deliver.
Those models are for general use. If you have business use case and data related to it you can finetune model for specific use that will outperform all of frontier models and run at fraction of cost.
what kinda hardware do you need to “run this on your desktop”?
You can run it on systems with as little as 32 GBs of RAM.
You don’t want to run these models in RAM. I started using them on an RTX 3060-12Gb VRAM, and quickly added another, for 24Gb RAM. I also have 64 Gb RAM. Now it works well. Not lightning fast, but quite useable. VRAM is the key.
Sure, but with a model of that size you are unloading layers in any case. I’m just saying you need at least 32 GBs of RAM at the minimum to be able to run it.
Kimi is better. Waiting for it to appear on Ollama Cloud.
So which one tried to cheat its way into the test?
Llama is one for sure
Which one of them made these overlapping bar labels?
Hey, a lot of people sucked making matplotlib plots before LLMs existed.
It’s a bit misleading.
Qwen 27B has way less “world knowledge” than GPT-5. Ask it random trivia without internet search access, and GPT would know waaay more.
This is generally true of small vs large models.
…But honestly, Qwen 27B is better at tool use or agentic stuff. It’s hyper optimized for just that and coding assistance, basically.
This is often true of old vs new. Most newer models have hyper focused on agents/coding, often to the detriment of other use cases.
Quantization for practically running Qwen 27V also has an impact. A off-the-shelf Q4_K_M is not the same as the unquantized weights in real-world use, or even an “optimized” quantization like a custom exl3.
If you run it with RAG (retrieval augmented generation), where the model includes web search, you get pretty decent results. I often include the phrase “search the web extensively for up to date information” in my prompts.
What version do you use and how do you run Qwen3.6? I’ve played around a bit with the Q4 version in LM studio +Zed, but I was not happy with the results. It looses track very often and often enters infinite loops or just stops…
I sort of get it because of you know you’re only using the model when you have Internet and want it to search then just let it.
I mean baking knowledge into a model isn’t really all that useful to begin with. Just download wikipedia locally and have it access it through tool use, it’s way more efficient and more accurate. And yeah, I find Q6 tends to be the sweet spot where it’s close enough to full 16 bit in performance, but doesn’t chew up too much memory.
This is a pretty good idea I hadn’t thought about. The whole wikipedia is about 37Gb.
[citation needed]
This is for qwen 3.5, not 3.6, but the vram requirements are a little out there for most:
https://willitrunai.com/blog/qwen-3-5-27b-vram-requirements
16 to 55GB…
I bought two RTX 3060 with 12 Gb each for a total of 24Gb VRAM. Sionce the cards are not top shelf anymore for gaming, they can be had used in the 200€ range, so my setup was 400€ over the cost of my existing PC. My PC, while not last gen was already pretty beefy, with a Ryzen 9 and 64 Gb RAM. I think 400€ to have a pretty good local AI setup is not bad at all.
I very much regret getting a 3070, with only 8GB of ram… Only reason I am hesitant about 3060s is that dual cards will use more power, and the economics are already pretty dicey when the hosted models are being sold below cost.
I can run both the 27B and 35B version of Qwen3.6 on my 10GB VRAM 3080, and they run ok. You don’t need to put the entire model in VRAM, even if it’s probably beneficial.
Full quantisation? I’ve only got a 8GB 3070, but I’ll give it a go
Edit: Tried the unsloth/qwen3.6 with llama.CPP, and it failed to allocate a 26GB Vulcan buffer and died. Dunno what magic your using, no luck for me though :(
I mean that’s high but not unreasonable, especially with unified ram machines
and it’s even lower with MTP https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF
Little outside my budget personally, but I’ve definitely been itching to pull the trigger on a nVidia P40
