In that example, it sounds like the train is responsible for driving forward, driving in reverse and stopping.
The train company would want to ensure a human has agreed in writing to be held accountable for foreseeable failures of the train driving system.
Sharing my definitions, more or less, in red:
The thing that should be impossible is for the train company to hire the owner’s drunk cousin to build a self driving train system, and when people get hurt and the company is taken to court, the company is allowed to apologize + blame software + do business as usual w/o fine/penalties.
I’m thinking even at the time enough stuff was (successfully, safely) automated that they would’ve agreed.
You want everyone at your company to know that if they get dragged in front of a judge, pointing fingers at silicon will not prove a winning strategy. Hence, a computer must never make a management decision!
In that example, it sounds like the train is responsible for driving forward, driving in reverse and stopping.
The train company would want to ensure a human has agreed in writing to be held accountable for foreseeable failures of the train driving system.
Sharing my definitions, more or less, in red:
The thing that should be impossible is for the train company to hire the owner’s drunk cousin to build a self driving train system, and when people get hurt and the company is taken to court, the company is allowed to apologize + blame software + do business as usual w/o fine/penalties.
I’m thinking even at the time enough stuff was (successfully, safely) automated that they would’ve agreed.
You want everyone at your company to know that if they get dragged in front of a judge, pointing fingers at silicon will not prove a winning strategy. Hence, a computer must never make a management decision!