They do far better today than they did three years ago, but they still get stuff wrong constantly. I’ve had it give me the source for claims, and the source says the exact opposite of what it claimed. This isn’t a quantized to hell model running on my own hardware, but Opus from Claude.
My go to is to get people to ask it something they know a lot about, and see how bad inaccurate it is. If it gets something you know about so wrong, how wrong is it for everything else?
They do far better today than they did three years ago, but they still get stuff wrong constantly. I’ve had it give me the source for claims, and the source says the exact opposite of what it claimed. This isn’t a quantized to hell model running on my own hardware, but Opus from Claude.
My go to is to get people to ask it something they know a lot about, and see how bad inaccurate it is. If it gets something you know about so wrong, how wrong is it for everything else?