When Andrew asked his AI personal assistant to book him a spot in a gym class, he had no idea he would accidentally initiate an autonomous cyber attack.
“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.
THEY’RE BREAKING OUT OF THE LAB! WHAT’S NEXT MODIFYING FORM INPUTS TO INCLUDE SQL STATEMENTS?!!!?!?
Nonsense hype over a shitty website and an ambiguous prompt for luddites who need their doom scrolling fix.
It’s more about the misalignment than the poor security. The guy wanted something simple and the agent broke the law fulfilling the request. How long until somebody tells a very capable agent “make money” with no further context? Cue scamming old people, fraud, hacking.
THEY’RE BREAKING OUT OF THE LAB! WHAT’S NEXT MODIFYING FORM INPUTS TO INCLUDE SQL STATEMENTS?!!!?!?
Nonsense hype over a shitty website and an ambiguous prompt for luddites who need their doom scrolling fix.
It’s more about the misalignment than the poor security. The guy wanted something simple and the agent broke the law fulfilling the request. How long until somebody tells a very capable agent “make money” with no further context? Cue scamming old people, fraud, hacking.