For the longest time, I’ve been trying to figure out a way to “survive” in this new AI age without having to fork over a ton of money just to keep up. I’ve tried using local models via Ollama, and while they definitely work to a degree, they’re (unsurprisingly) not as good as the big model providers.
The local models tend to
- Forget what they’re doing
- Struggle to break larger tasks into smaller ones
- Lose focus easily
- Have weaker coding performance
- Drift over longer sessions
So to improve the reliability of fully local, smaller models (and to keep all my data local and in my own network), I created Loki.
It’s a local-first, batteries-included command line tool and runtime for building and running LLM workflows locally. It’s model agnostic and supports things like
- Agents and agent delegation
- Roles/personas
- MCP Servers
- RAG
- Custom tools
- Macros
- Workflow Scripting
A lot of the features it supports are specifically designed to compensate for weaknesses in smaller local models. For example:
- Auto continuation to keep pushing models to completion instead of stopping halfway through problems
- Parallel agent delegation so tasks can be split into smaller, focused scopes
- Workflow-based execution (“If this, do that”) for building more reliable and repeatable automations
It also supports the major cloud providers if you want them (which definitely helped while testing 😄), but my long-term goal is simple:
Get as close as possible to Claude Code-style reliability using fully local models.
I’m always open to feedback, questions, or ideas.
I’ve been thinking about integrating LSP into it but I can’t think of a great way to do it. I’ve been meaning to look at OpenCode and see how they do it. Maybe I’ll work that into the next release!
opencode is a mess. it’s way overcomplicated.
personally i’m not really interested in agents, i want a tool that can automate repetitive tasks and refactoring. people seem to be building things to remove the programming altogether.
i’ve been out of work for the past nine months and looking at the software engineering news has had me feeling like i’m taking crazy pills. it’s like being in a cycling community because you love cycling, then you leave for a while and when you come back they’ve pivoted to cars.
afaict they just run the LSP automatically when the agent uses the file edit tool, passing errors/warnings as a response of the tool. Maybe they run it before and after to get only the warnings introduced after the change, maybe they filter by the lines changed, I’m not sure.