SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 1 day agoDo you host your own AI?message-squaremessage-square181fedilinkarrow-up1158arrow-down135file-text
arrow-up1123arrow-down1message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 1 day agomessage-square181fedilinkfile-text
minus-squaree0qdk@reddthat.comlinkfedilinkEnglisharrow-up3·22 hours agoIf you just pulled the default version of qwen3.5 from ollama’s repo you downloaded a mediocre one that only uses ~6GB. Check ollama show qwen3.5 and see if you get something like this in the result: Model architecture qwen35 parameters 9.7B context length 262144 embedding length 4096 quantization Q4_K_M This is the default version I got when I first tried using ollama without any experience. It worked, but it’s a heavily quantized, lower parameter version of the model – i.e. it’s pretty dumb – compared to what you can actually run on your hardware.
minus-squareSteve@startrek.websitelinkfedilinkEnglisharrow-up2·10 hours agoI will check it later. I loaded whichever one cluade suggested lol
If you just pulled the default version of qwen3.5 from ollama’s repo you downloaded a mediocre one that only uses ~6GB.
Check
ollama show qwen3.5and see if you get something like this in the result:Model architecture qwen35 parameters 9.7B context length 262144 embedding length 4096 quantization Q4_K_MThis is the default version I got when I first tried using ollama without any experience. It worked, but it’s a heavily quantized, lower parameter version of the model – i.e. it’s pretty dumb – compared to what you can actually run on your hardware.
I will check it later. I loaded whichever one cluade suggested lol