I have a simple slow model running on CPU in my cluster for karakeep. I’ve tried running a variety of models on my 7900XT but even with 16GB their performance just isn’t there. My new work m5 Mac book with 48GB of ram is the first time I’ve seen usable performance for local models and it has been pretty impressive.
I have a simple slow model running on CPU in my cluster for karakeep. I’ve tried running a variety of models on my 7900XT but even with 16GB their performance just isn’t there. My new work m5 Mac book with 48GB of ram is the first time I’ve seen usable performance for local models and it has been pretty impressive.