I’ve been wanting to do this, even as a testbed. I just want computers to do cool shit, and am getting tired of being a Linux sysadmin in my free time. How large of a model do you have assigned to this?
I was fortunate to have bought my graphics card almost a year ago, and for a nice discount (a Black Friday sale, IIRC). I’d love to buy another, but they now cost >2x the price I paid.
FWIW, you can probably run it on CPU if you have enough RAM. I assume it will be really slow though.
Might also be worth trying Qwen3.6 35B-A3B. It’s not as “smart” as Qwen3.8 27B, but can be much faster on limited resources.
I’ve been wanting to do this, even as a testbed. I just want computers to do cool shit, and am getting tired of being a Linux sysadmin in my free time. How large of a model do you have assigned to this?
Just Qwen3.8-27B. That little model is surprisingly smart.
NixOS also gives you the ability to roll back to previous config versions. I have not needed to do that (yet), but it reduces some risk.
“Just” and a 27B model don’t exactly belong in the same sentence if you ask my wallet. 💸
Yeahhh sorry 😕
I was fortunate to have bought my graphics card almost a year ago, and for a nice discount (a Black Friday sale, IIRC). I’d love to buy another, but they now cost >2x the price I paid.
FWIW, you can probably run it on CPU if you have enough RAM. I assume it will be really slow though.
Might also be worth trying Qwen3.6 35B-A3B. It’s not as “smart” as Qwen3.8 27B, but can be much faster on limited resources.