Fun experiment: Ask Google if there are more stars in the solar system than grains of sand in a glass of water. See the AI confidently say “yes” and then refresh the query and see it confidently say “no”.
I have experimented and can very often get AI to give whatever answer I want (i.e. on a yes/no) by subtly changing the query. Super easy to manipulate the results.
The same is predictably true in research, meaning a lot of academic research being produced at the moment is complete crap.
https://arxiv.org/abs/2509.08825
I don’t know if I’d say “a lot,” but I think you are correct to say “some.” We don’t need to suddenly start flinging around generalized invectives towards science, especially when there’s already a concerted effort by conspiracy theorists and bad actors to convince people to reject science out of hand.
True, there’s a lot of good research being produced as well. There’s just an insane amount of academic work being produced in general, both good and bad.
Yes, and I passed that research paper onto someone who works with researchers who think LLMs are the best new invention for technical writing. It’s going to be a constant battle, even amongst people who should know better.
Fun experiment: Ask Google if there are more stars in the solar system than grains of sand in a glass of water. See the AI confidently say “yes” and then refresh the query and see it confidently say “no”.
I have experimented and can very often get AI to give whatever answer I want (i.e. on a yes/no) by subtly changing the query. Super easy to manipulate the results.
The same is predictably true in research, meaning a lot of academic research being produced at the moment is complete crap. https://arxiv.org/abs/2509.08825
I don’t know if I’d say “a lot,” but I think you are correct to say “some.” We don’t need to suddenly start flinging around generalized invectives towards science, especially when there’s already a concerted effort by conspiracy theorists and bad actors to convince people to reject science out of hand.
True, there’s a lot of good research being produced as well. There’s just an insane amount of academic work being produced in general, both good and bad.
Yes, and I passed that research paper onto someone who works with researchers who think LLMs are the best new invention for technical writing. It’s going to be a constant battle, even amongst people who should know better.
Yes, it is annoying to see text written that is confidently asserting nonsense