Welp. My Forgejo instance got popped with an RCE two days ago by CVE-2026-60004. Luckily, I noticed the following morning and had the day free to figure out what happened. Let’s dive in!
As a homelab enthusiast, I found this a very interesting post. Here are my take aways from the post that I’m implementing myself:
- Miner detection. I’ve updated monitoring rules to now watch the CPU on my hosts. If the same thing happened to me I would not have been alerted at all as I’m doing simple up / down monitoring. Fixed.
- Access logging. I turned on access logging for my homelab Caddy instances.
- Log retention. I have increased the amount and retention of my logging. The hope is this will help me reconstruct what happened after a breach.
- Logs offsite. The VPS access logs now ride along with the normal backup process, which runs hourly. The homelab side still only gets caught by the weekly VM backup, so that’s next.
- Closed an open signup. My webtrees instance (genealogy) had self registration enabled, which is the same door this guy got hit through. Oops. Fixed.
- Built a tool.
log-inventory.sh, so “could I actually reconstruct what happened” is a command I run instead of a thing I assume.
Yiiiiiikes. This is why I’m gonna be stuck on tailscale or another VPN only access for quite a while. I know it’s not perfect, but exposing ports on today’s Internet is scary.
I know I don’t have the mental overhead to secure literally everything, and have a solid contingency for when some botnet armed with a zero-day inevitably breaks it.
Really glad this person wrote this write up about how that happens and what it looks like though. It’s REALLY interesting and I learned a lot!
Check out CloudFlare tunnels. I created a list of e-mails, and only the people who enter the confirmation code sent to those emails get into my home server.
About the only recent even I’ve had was when I was sitting at my desk reading an article. I noticed the mouse slightly move, but I hand no hand on the mouse. I sit there watching it, not manually moving the mouse or typing on the keyboard. It really freaked me out. Turns out, my old desk pad had a slight curl towards the edge the mouse was at, and the curl would make the mouse sense movement ever so slightly. I deployed a new desk pad, and things are back to normal. Whew!
My server got hacked once and was mining crypto! I noticed it when I heard the fans were at 100% when walking passed the room lol. Turns out, qbittorrent-nox used UPNP for its web admin endpoint enabled by default and qbittorrent can do many things like run custom scripts. My mistake was not changing its login credentials. Exposed my full system. Live and learn.
UPNP is a security nightmare, better to never enable it at all or if you must only for those networks where you need it.
I just recently went through a much more benign, but scary nonetheless version of this.
I realized that my Ansible directory, that I had made public on GitHub to share as an example to some folks, had secrets committed and pushed.
It was the direct URL and credentials of an app I developed to store non-PII customer data. Now, it wouldn’t be the end of the world if someone noticed this and scraped the data, but it wouldn’t be good, either.
Luckily, I have Caddy access logs, and it appears no one ever accessed it.
So I pulled the secrets out of the Ansible directory and made the repo private, I rotated the credentials, and installed Crowdsec to monitor Caddy access logs and ban bad actors.
I only noticed the secrets because I had just setup Authelia as an OAuth2 provider for my homelab, and I was adding it to my backup scripts.
Ayy, just went through the same troubles with Authelia earlier this week.
Only 4 services I could conmect ao far but still neat to have :)
And detected mostly because the automated script kiddie set up a cryptominer to generate $0.0001
There are way more valuable and destructive ways once you have RCE like:
- Searching for committed API keys “anyway my repo is private”
- Ransomware in case there were no backups
I’d suggest hobbling those log links that point to active payloads etc - usually best practice when writing this stiff up, so someone doesn’t accidentally click on them…
I hate providers not offering me a latest tag. Is there any valid reason not to do that?
Even worse, super stupid version tags such as MinIO is using:
RELEASE.2025-09-07T16-13-09Z-cpuv1(I wish I made that up)My guess, auto updates for major versions is not good practice. Installations potentially corrupt and users come screaming to the project maintainers instead of searching the blame by themselfe.
forgejo defined the lifecycle for every version so just add a calendar entry for when the supports runs out and then update the 3 characters in the docker compose file.
I can get that and I also prefer sticking to major versions. However, I am chaotic and easily have 50 different docker images, checking all the release cycle of all of those gets exhausting.
I believe I need to start generating notifications if a tag has not received an update for more than 30d.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters Git Popular version control system, primarily for code SSH Secure Shell for remote terminal access TLS Transport Layer Security, supersedes SSL VPN Virtual Private Network VPS Virtual Private Server (opposed to shared hosting)
[Thread #97 for this comm, first seen 7th Sep 2026, 20:40] [FAQ] [Full list] [Contact] [Source code]
Firewalling containers’ outbound access seems to be rare but another powerful layer of protection
Yeah it’s useful and I’ve been using it for my home lab, but too many of my containers want outbound connections to the Internet. Like Forgejo needs :443 to be able to mirror or remote push repositories, so either I have to identify and allow list github.com, gitlab.com, etc. or end up allowing 0.0.0.0/0:443 which is not very strong protection.
I’d love for some kind of CNI or network plugin that filtered based on domain name.
Envoy is what i use, or squid is an option
The real takeaway is to run your containers ALL as read-only. There’s no way to run executables like this as read-only. They’d download to /tmp but not have executable access and womp womp.
Many things aren’t read-only friendly, but so far I’ve found all containers can be beat into read-only mode! (Heimdall is one of the worst, shout out to it)
Good write up! Since you’re using Komodo already, are your compose stacks in git? Because if they are, check the renovate bot. It will read your files with their pinned image tags, check upstream for newer tags, and creates MRs (can also be set to auto merge) to update. Can be configured per stack.
What’s komodo? I’m thinking of starting a home server so learning how to harden/automate updates (which is what it sounds like you’re saying) is what I want to learn.
Not updating stuff + Public sign ups enabled (did you even read the setup guide?) + Unrestricted internet access
Yeah I wonder what could possibly go wrong…
But great writeup
At least he got to learn from it. Never waste a good disaster.
And sharing it with peeps who haven’t encountered these vulnerabilities yet. We are all trying to improve and reading this is better than doomscrolling.
Also let’s be honest, it was entertaining.
Adventure is when trouble happened to somebody else! :p
Thanks for this! I didn’t understand any of it, but I hope to eventually.
OIDC and zero signups for everything. I’ve been very happy with Pomerium (ZTNA) + Keycloak for all my public facing apps, although the free version doesn’t have any client signals which I was hoping for when I starteed.
My project this week is integrating the app logging I have into Crowdsec and start having Crowdsec do more analysis.
Nice write up, also glad to see someone using bear blog in the field! Adding your blog to my RSS reader!
