lemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 1 個月前Opensource AI Must Winopensourceaimustwin.comexternal-linkmessage-square111fedilinkarrow-up1284arrow-down120cross-posted to: [email protected]
arrow-up1264arrow-down1external-linkOpensource AI Must Winopensourceaimustwin.comlemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 1 個月前message-square111fedilinkcross-posted to: [email protected]
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up1·1 個月前I mean if that’s all that would be loaded in memory, sure.
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4arrow-down1·1 個月前I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up2·1 個月前I was talking about combined system RAM. People often overestimate what the average system specs are.
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4arrow-down1·1 個月前If the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM
I mean if that’s all that would be loaded in memory, sure.
I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
I was talking about combined system RAM. People often overestimate what the average system specs are.
If the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM