There was a dedicated person on call, but it happened to be when they were away.
The Konsole was left running from a local access, with a while true loop of a service status command. When that service was stopped later, the while loop started rerunning the script every second, filling the buffer with error messages.
The tab was then killed remotely, but the Konsole window left running. Process ram usage went down but the file remained on tmpfs, which is not counted as ram usage so wasn’t noticed.
Then it took some time to notice the ram usage mismatch so noone thought of that konsole incident.
Ah that makes sense. I had considered it might be a server but you mentioned a Konsole tab so my mind decided it must have been local machine.
Crazy it took 2 days to restart the server!
There was a dedicated person on call, but it happened to be when they were away.
The Konsole was left running from a local access, with a while true loop of a service status command. When that service was stopped later, the while loop started rerunning the script every second, filling the buffer with error messages.
The tab was then killed remotely, but the Konsole window left running. Process ram usage went down but the file remained on tmpfs, which is not counted as ram usage so wasn’t noticed.
Then it took some time to notice the ram usage mismatch so noone thought of that konsole incident.