So…with all this openclaw stuff, I was wondering, what’s the FOSS status for something to run locally? Can I get my own locally run agent to which I can ask to perform simple tasks (go and find this, download that, summarize an article) or things like this? I’m just kinda curious about all of this.
Thanks!
No one has mentioned Open Web UI, which is part of this landscape.
Open Web UI is the chat interface you use to interact with a model. I haven’t really dug into much of the functionality beyond simple chat, but there’s thousands of community plugins for web search and similar. You can also create knowledge bases and attach them to queries. For example if I have a bunch of policy and procedure documents from my work, I can create a knowledge base and ask the LLM to create new policies in that context.
You can configure it to work with ollama, which allows you to run LLMs from huggingface.co and similar on your own hardware.
However, in my own case I just don’t have anything resembling a modern powerful GPU, so I don’t run ollama locally. You can use a paid account at huggingface.co and use their API to do the inference (running the models). Not all LLMs are available this way but certainly many are.
More recently I’ve discovered that OVH, (a french bare-metal host I’ve used for years) provides an inference API for a half dozen models, and I’ve found this to be blistering fast compared to huggingface.