☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 13 hours agoGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.mlimagemessage-square29fedilinkarrow-up172arrow-down111
arrow-up161arrow-down1imageGPT-5, the world best model just 1 year ago, is today inferior to Qwen3.6 27B that you can run on your desktoplemmy.ml☆ Yσɠƚԋσʂ ☆@lemmy.ml to Technology@lemmy.mlEnglish · 13 hours agomessage-square29fedilink
minus-squareelucubra@sopuli.xyzlinkfedilinkarrow-up1·2 hours agoYou don’t want to run these models in RAM. I started using them on an RTX 3060-12Gb VRAM, and quickly added another, for 24Gb RAM. I also have 64 Gb RAM. Now it works well. Not lightning fast, but quite useable. VRAM is the key.
minus-squareHubi@feddit.orglinkfedilinkarrow-up1·edit-248 minutes agoSure, but with a model of that size you are unloading layers in any case. I’m just saying you need 32 GBs of RAM at the minimum to be able to run it.
You don’t want to run these models in RAM. I started using them on an RTX 3060-12Gb VRAM, and quickly added another, for 24Gb RAM. I also have 64 Gb RAM. Now it works well. Not lightning fast, but quite useable. VRAM is the key.
Sure, but with a model of that size you are unloading layers in any case. I’m just saying you need 32 GBs of RAM at the minimum to be able to run it.