As funny as this is (and I do find it funny), it’s also concerning on a wider level. A good number of people trust these AI summaries; they shouldn’t, but they do. And if it’s this easy to poison the AIs, imagine how easy it is for someone with an actual agenda to mislead people in ways that aren’t as fantastical and quickly spotted.
I seem to recall reading recently that a court in Germany wanted to hold Google accountable for the content of its AI summaries. (Someone correct me if I’m wrong, please.) If companies are going to shove these models in people’s faces they should absolutely be responsible for the results. If your model can’t tell fact from fiction, stop publishing - and promoting - it as fact.
And if it’s this easy to poison the AIs, imagine how easy it is for someone with an actual agenda to mislead people in ways that aren’t as fantastical and quickly spotted.
Equally concerning is that these systems are now seeing use in a range of things. There are lawyers who use it to file suits when they shouldn’t be, and a US lawmaker was recently found to be using AI to draft laws. What happens when things like that make it into the models training data, rather than just being pulled in by RAG/web tools? They’d become part of the base knowledge of all the models of that line going forward.
It’s funny when it’s outlandish. The question becomes what happens when it isn’t? Even without an agenda, what happens when it cites an outdated/incorrect source, or assumes that someone making a joke was correct, and ends up drawing from that when filling a lawsuit/drafting a law?
Thank you for finding the link! I’ve no doubt Google will fight for as long as they can, but hopefully the German courts will hold their ground.
I’m far from an expert, but I feel like this is one of the limiting factors of LLMs - they have no sense of broader context. Truth vs. lie, outdated info vs. something that’s old but still correct… I’m not sure there’s ever going to be an LLM (at least one built in the way they are now) that will be good at actually producing correct responses. Maybe one day we’ll find a new way of achieving that goal, but I suspect what we’re seeing now isn’t going to be it.
As funny as this is (and I do find it funny), it’s also concerning on a wider level. A good number of people trust these AI summaries; they shouldn’t, but they do. And if it’s this easy to poison the AIs, imagine how easy it is for someone with an actual agenda to mislead people in ways that aren’t as fantastical and quickly spotted.
I seem to recall reading recently that a court in Germany wanted to hold Google accountable for the content of its AI summaries. (Someone correct me if I’m wrong, please.) If companies are going to shove these models in people’s faces they should absolutely be responsible for the results. If your model can’t tell fact from fiction, stop publishing - and promoting - it as fact.
The case recently resolved in the plaintiff’s favour. though Google intends to appeal, so it’s up in the air how things go.
Equally concerning is that these systems are now seeing use in a range of things. There are lawyers who use it to file suits when they shouldn’t be, and a US lawmaker was recently found to be using AI to draft laws. What happens when things like that make it into the models training data, rather than just being pulled in by RAG/web tools? They’d become part of the base knowledge of all the models of that line going forward.
It’s funny when it’s outlandish. The question becomes what happens when it isn’t? Even without an agenda, what happens when it cites an outdated/incorrect source, or assumes that someone making a joke was correct, and ends up drawing from that when filling a lawsuit/drafting a law?
Thank you for finding the link! I’ve no doubt Google will fight for as long as they can, but hopefully the German courts will hold their ground.
I’m far from an expert, but I feel like this is one of the limiting factors of LLMs - they have no sense of broader context. Truth vs. lie, outdated info vs. something that’s old but still correct… I’m not sure there’s ever going to be an LLM (at least one built in the way they are now) that will be good at actually producing correct responses. Maybe one day we’ll find a new way of achieving that goal, but I suspect what we’re seeing now isn’t going to be it.
And DuckDuckGo AI is exceptionally bad. Like, other AI is bad too, but DuckDuckGo’s version is even behind the pack.