btw this is kinda an outdated viewpoint. what is even meant by “management decision”? is it decisions such as “does the train drive forward, or not?” because if that’s what’s meant, then yes, very much can machines make these kind of decisions. that’s the whole point of studying real-time systems. they require that they can handle every possible input data case and respond to it meaningfully within a fixed maximum response time. it has nothing to do with “holding the decision maker accountable”.
In that example, it sounds like the train is responsible for driving forward, driving in reverse and stopping.
The train company would want to ensure a human has agreed in writing to be held accountable for foreseeable failures of the train driving system.
Sharing my definitions, more or less, in red:
The thing that should be impossible is for the train company to hire the owner’s drunk cousin to build a self driving train system, and when people get hurt and the company is taken to court, the company is allowed to apologize + blame software + do business as usual w/o fine/penalties.
I’m thinking even at the time enough stuff was (successfully, safely) automated that they would’ve agreed.
You want everyone at your company to know that if they get dragged in front of a judge, pointing fingers at silicon will not prove a winning strategy. Hence, a computer must never make a management decision!
“AI told me to bomb this place’s which just happens to be just a school full of terrorist muslim children. I’m sowwy for this war crime and I promise not to do it again by improving the model. UwU”
btw this is kinda an outdated viewpoint. what is even meant by “management decision”? is it decisions such as “does the train drive forward, or not?” because if that’s what’s meant, then yes, very much can machines make these kind of decisions. that’s the whole point of studying real-time systems. they require that they can handle every possible input data case and respond to it meaningfully within a fixed maximum response time. it has nothing to do with “holding the decision maker accountable”.
In that example, it sounds like the train is responsible for driving forward, driving in reverse and stopping.
The train company would want to ensure a human has agreed in writing to be held accountable for foreseeable failures of the train driving system.
Sharing my definitions, more or less, in red:
The thing that should be impossible is for the train company to hire the owner’s drunk cousin to build a self driving train system, and when people get hurt and the company is taken to court, the company is allowed to apologize + blame software + do business as usual w/o fine/penalties.
I’m thinking even at the time enough stuff was (successfully, safely) automated that they would’ve agreed.
You want everyone at your company to know that if they get dragged in front of a judge, pointing fingers at silicon will not prove a winning strategy. Hence, a computer must never make a management decision!
“AI told me to bomb this place’s which just happens to be just a school full of terrorist muslim children. I’m sowwy for this war crime and I promise not to do it again by improving the model. UwU”
Literally just happened last year: https://www.theguardian.com/news/2026/mar/26/ai-got-the-blame-for-the-iran-school-bombing-the-truth-is-far-more-worrying
Computers are just scapegoats in avoiding accountability