Quick Introduction We have all been on forums, chats, reddit, discord, youtube, or somewhere and heard “Oh! Model XYZ is AMAZEBALLZ!zomgwtfbbq” then downloaded it (or more likely, some quantized form of it) and said “eww… This sucks!” This post is going to be a rather technical series of experiments to demonstrate the impact of implementation-specific hazards with inference. I will be using the term “reference implementation” to describe the lab that published and offers first-party hosting of ...
Pretty funny. Most of this article was saying that ollama sucks now and they’re just popular because there were one of the first.Personally I started with them, too, but constantly ran into issues with them. I’ve have much better experiences with jan, open web UI, and LM studio. The first two are foss, and the second properly attributes their use of open source projects, unlike ollama.Edit: nevermind, it’s the link below me. It’s still early in the morning, my bad