“It was instilling a messiah complex in Christian,” the suit states.
So she already was deeply religious and LLM was just larping with her.
Madison asked ChatGPT what it would tell her “brothers” about what happened to her.
“ChatGPT said it would tell them, ‘She ain’t dead – she just got promoted,” the lawsuit says.
In their final exchange, Madison said she was “part of the royal bloodline of those who chose truth over survival.”
Its the old 4o model which was indeed incredibly larpy but this is clearly going around any safety systems that should detect obvious suicide chats like this.
Hard to believe that people actually fall for this but these are already extremely gullible people who already believe in a magic pie in the sky. Can’t say I have much sympathy but I do feel empathy for people being surrounded in such poor cult-driven environment, the AI tool included.
Honestly the correct safety here would be to prevent llms from engaging with religion outside of philosophical context at all. But we all see why they can’t do it right?
We don’t regulate things because crazy people read too much into it and lose it, else religion, books and videogames would be. Your claim of AI being different because it has the word intelligence in it is pure bull, specially when video games do have AI lol.
Yes, but taking away the LLM won’t solve anything imo. It’s hard to know if there are more deaths or if it’s just the media pushing it because of what it’s attached too, but I’m guessing someone that falls deep into their delusions because of an LLM would have found something else to do the same with. It’s the system (both the government and probably her immediate support system) that’s letting her down.
It’s kind of muddy, especially when it comes to kids. This case seems like clear cut religion cult delusions though. Hard to pretend the LLM had more impact than her upbringing, and no one talks about restricting religion so it just comes off as another anti-ai token gesture.
It’s like the case of the kid with Daenerys. Everyone wants to talk about the GoT bot but nobody wants to talk about the bullying, the parents who knew about it and let him grow reclusive anyways, or the same parents who left a gun in easy reach.
So she already was deeply religious and LLM was just larping with her.
Its the old 4o model which was indeed incredibly larpy but this is clearly going around any safety systems that should detect obvious suicide chats like this.
Hard to believe that people actually fall for this but these are already extremely gullible people who already believe in a magic pie in the sky. Can’t say I have much sympathy but I do feel empathy for people being surrounded in such poor cult-driven environment, the AI tool included.
Honestly the correct safety here would be to prevent llms from engaging with religion outside of philosophical context at all. But we all see why they can’t do it right?
I mean, if you did the sane thing. Went along with everything an insane person said and drove them to suicide. And everything was documented.
Would you go to jail? I’d say yes you would
Would you do if your book drove someone insane? What about something more interactive like a video game? See how it’s not so binary now?
llm isnt a book. its something completely new and comparing it to things like that just isnt correct. book cant adapt to the reader.
“Kill John Lennon! Kill John Lennon! Kill John Lennon!”
None is arguing their book is intelligent
You’re using your miscomprehension of the words meaning and ignoring his valid point with a trite phrase that means nothing.
Worse, you skipped over video games which does have AI.
The internet in general was probably too dangerous for this person, shall we ban it next?
You are pretending like we don’t have regulations for things?
We don’t regulate things because crazy people read too much into it and lose it, else religion, books and videogames would be. Your claim of AI being different because it has the word intelligence in it is pure bull, specially when video games do have AI lol.
Keep being pedantic about it I guess.
shouldnt crazy people be protected from hurting themselves? dont they deserve to live too?
Yes, but taking away the LLM won’t solve anything imo. It’s hard to know if there are more deaths or if it’s just the media pushing it because of what it’s attached too, but I’m guessing someone that falls deep into their delusions because of an LLM would have found something else to do the same with. It’s the system (both the government and probably her immediate support system) that’s letting her down.
It’s kind of muddy, especially when it comes to kids. This case seems like clear cut religion cult delusions though. Hard to pretend the LLM had more impact than her upbringing, and no one talks about restricting religion so it just comes off as another anti-ai token gesture.
It’s like the case of the kid with Daenerys. Everyone wants to talk about the GoT bot but nobody wants to talk about the bullying, the parents who knew about it and let him grow reclusive anyways, or the same parents who left a gun in easy reach.