• PeterPoopshit@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      edit-2
      1 year ago

      Use llama cpp. It uses cpu so you don’t have to spend $10k just to get a graphics card that meets the minimum requirements. I run it on a shitty 3.0ghz Amd 8300 FX and it runs ok. Most people probably have better computers than that.

      Note that gpt4all runs on top of llama cpp and despite gpt4all having a gui, it isn’t any easier to use than llamacpp so you might as well use the one with less bloat. Just remember if something isn’t working on llamacpp, it’s also going to not work in exactly the same way on gpt4all.

        • NotMyOldRedditName@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          edit-2
          1 year ago

          Check this out

          https://github.com/oobabooga/text-generation-webui

          It has a one click installer and can use llama.cpp

          From there you can download models and try things out.

          If you don’t have a really good graphics card, maybe start with 7b models. Then you can try 13b and compare performance and results.

          Llama.cpp will spread the load over the cpu and as much gpu as you have available (indicated by layers that you can set on a slider)