This is the thing about LLMs and their users I genuinely don’t get. Like, objectively, being able to input plain speech into a computer and get an almost immediate, detailed, 95% accurate answer is incredible. Genuine technological marvel.
But 95% accuracy is functionally useless for all but the most basic, unimportant tasks. Outside of a coding context (a big exception, to be clear), are people really satisfied with that?
Let’s be honest, most sources online are not 100% accurate. Even before LLMs, find any random article about something you’re an expert in, and there was a good chance you’d spot some inaccuracies. Most of the time, those inaccuracies aren’t a big deal, and when they are, you should be consulting an expert for those.
For me personally, I would rate AI accuracy at about 75%. And for some of my use cases, that’s a pretty decent starting point. Things with no consequences, like trying to quickly find the relevant console command for the game I’m playing. I can get instant feedback on whether it gave me correct info, and if not then I hit Google.
I would never trust AI, at least in its current state, to tell me the truth for anything of consequence.
The majority of human history at all levels of science, philosophy, and mathematics. The best you could ever hope for was 95%. Hell you were lucky to break 90% accuracy.
Because the only people you could ask questions was the people in the room with you right then and there or the people you could actively physically go ask.
If you were extremely lucky you could write to someone who MIGHT know more. Or may there’s a book about it that someone knows exists.
95% can and frequently is very much over kill for almost everything anyone will need for their day to day life for random trivia or even most important work.
Very few fields need more than that, and few still is it a matter of life or death.
The part that people fuck up is not being open to learning more and being corrected.
If you can confirm it, it doesn’t matter what percentage is correct.
When solving computer problems, using an LLM to guide you to the correct answer doesn’t even need 95% accuracy. Every step it gives you have to execute manually, so if something is hallucinated you will notice really quickly. And if by the end your problem is solved, then all the hallucinations didn’t matter.
This is the thing about LLMs and their users I genuinely don’t get. Like, objectively, being able to input plain speech into a computer and get an almost immediate, detailed, 95% accurate answer is incredible. Genuine technological marvel.
But 95% accuracy is functionally useless for all but the most basic, unimportant tasks. Outside of a coding context (a big exception, to be clear), are people really satisfied with that?
Let’s be honest, most sources online are not 100% accurate. Even before LLMs, find any random article about something you’re an expert in, and there was a good chance you’d spot some inaccuracies. Most of the time, those inaccuracies aren’t a big deal, and when they are, you should be consulting an expert for those.
For me personally, I would rate AI accuracy at about 75%. And for some of my use cases, that’s a pretty decent starting point. Things with no consequences, like trying to quickly find the relevant console command for the game I’m playing. I can get instant feedback on whether it gave me correct info, and if not then I hit Google.
I would never trust AI, at least in its current state, to tell me the truth for anything of consequence.
Most scientific reviews cite papers in which only the title was read, and often the titles are not justified in the papers.
The majority of human history at all levels of science, philosophy, and mathematics. The best you could ever hope for was 95%. Hell you were lucky to break 90% accuracy.
Because the only people you could ask questions was the people in the room with you right then and there or the people you could actively physically go ask.
If you were extremely lucky you could write to someone who MIGHT know more. Or may there’s a book about it that someone knows exists.
95% can and frequently is very much over kill for almost everything anyone will need for their day to day life for random trivia or even most important work.
Very few fields need more than that, and few still is it a matter of life or death.
The part that people fuck up is not being open to learning more and being corrected.
If you can confirm it, it doesn’t matter what percentage is correct.
When solving computer problems, using an LLM to guide you to the correct answer doesn’t even need 95% accuracy. Every step it gives you have to execute manually, so if something is hallucinated you will notice really quickly. And if by the end your problem is solved, then all the hallucinations didn’t matter.
You could do just as well googling Stack Overflow back when Google wasn’t shit.
But now both SO and Google are shit…