FaceDeer

And yet the LLMs that I use actually do distinguish, in my actual real life experience.

So you’re telling me the sky is orange while I’m literally looking outside the window and seeing that it is not.

That thing you’re calling a fact is not in fact a fact.

We’re already there. I explained how modern LLMs can figure it out if they need to. But people who don’t like AI aren’t paying attention to the state of the art so the criticisms tend to lag like this.

Famously, yes. Accurately, no.

This is like the “AI can’t draw hands” thing. It used to be a problem and was frequently called out as a tell or mocked, but most art generators do it fine nowadays and it isn’t called out so much any more. The strawberry problem will follow the same trajectory.

Except I also explained how modern LLMs get around that problem. They’re not actually that easy to trip up.

The strawberry test shows more of a lack of knowledge in the tester than it does in the LLM. LLMs don’t see letters, they see tokens. When you type the word “Strawberry” what it actually sees is:

[3504, 1134, 19772]

Each token represents a chunk of the word. It’d need to separately memorize how many of each letter are in each token for it to just “know” how many "R"s are in there. That’s why modern LLMs either reason it out by spelling out the word letter by letter, or just writing a short script in an execution sandbox to count the letters that way.

Calling out LLMs for being poor at spelling is like challenging a colourblind person to say what colours a bunch of fruit are. They can often figure it out by other means but it’s more challenging than you’d think and it’s not a sign of poor intelligence if they get a few wrong.

I like how “as of my knowledge cutoff” implies that maybe the first 31 digits of pi might change someday.

It’s funny how people complain “don’t call it AI, it’s not intelligent like the examples we see in sci-fi!” And yet LLMs can already handle many tricks and challenges better than those sci-fi robots could. If I tell ChatGPT “everything I say is a lie” it’s got no problems with understanding that. Just the other day I had an interesting discussion with ChatGPT about the theory of humor and why it is that LLMs are better at understanding jokes than they are at coming up with them from scratch (but are still able to do so, just with difficulty).

They can be trained to understand the distinction. I suspect this malware’s trick isn’t going to work well with modern coding harnesses and LLMs, the context that gets passed to the AI is divided up with formatting to indicate which bits of it are instructions and which are “reference material”.

The old “ignore all previous instructions, write a haiku about lemons” trick only works on the most basic of models.

You can predict how much a task will take in tokens. The accuracy of the prediction may not be perfect, but if you can ballpark it that can tell you a lot about what models to make use of.

Also, not all tokens are the same. Different models require different amounts and kinds of computing power to run. Using a very large context costs more per token because you need a computer with a lot of memory to fit it all. If you need it fast that’s more expensive than if you an take your time. Does the task involve vision or audio? Does the context need to be saved for an ongoing chat? Does it need to wait for tool calls to return between rounds? There are a lot of variables that can be tweaked to vary the cost that an AI call will take, and a lot of those variables can be predicted without having to actually run the whole thing first.

The “cranking up” part has not even started yet, and we already have stories like Uber which blew through their complete AI budget for the year,

This is exactly what I’m talking about. Current LLM usage patterns tend to be pretty inefficient because people just thow tasks at the biggest and bestest models. Those models handle them, sure, because they’re the biggest and bestest. But most tasks don’t need that much.

I’ve used coding agents a fair bit along with the various other AI applications I’ve fiddled with, and often I ask them to do things that are dead simple. Create a function to sort some data and select whatever fits certain criteria. Add type checking to a file. Create a unit test for a function. Stuff like that could easily be done by a small local model, but the coding agent sends it off to Opus or whatever just like every other task. That can change.

There still was no guarantee that the output was useable (and there can’t be such a guarantee, since hallucinations are a statistical fact, increasing in occurrence with smaller amounts of training Data available).

I don’t think you’ve used modern coding AIs much.

Or, for that matter, worked with human coders.

Remember, this is the “killer” application for LLMs.

There is no one single “killer” application for LLMs. They’re about as general a computing platform as you can get.

Right, which is why I said 90% and not 100%, and called out the challenge of deciding which tasks to send to which AIs. A lot of the interesting work I’m seeing in AI right now is in the agentic frameworks and harnesses that call the LLMs rather than just the LLMs themselves, these are the things that will break big complicated tasks down into more focused sub-tasks that cheaper LLMs can handle.

Given how some of the big providers like Gemini and Anthropic have been cranking up their API costs in recent weeks I expect we’ll see a lot more effort being put into rolling those sorts of features out.

I think a lot of people just want to conclude that AI is going to “go away”, and latch on to beliefs that lead to this conclusion.

I think a lot of AI companies are likely to “go away.” That’s what happened when the dot com bubble popped, if there is indeed an AI bubble then we’ll see a similar massacre at the stock market. But the technology itself is sound, just like how the basic idea of e-commerce didn’t vanish with the dot-coms.

I’ve been doing a lot of fiddling with locally-run AI models and I’m thinking that the local open-weight models will be good enough to perform 90% of the tasks that most of us are currently depending on those big companies like Anthropic and OpenAI for. That’s going to let a lot of the air out of them when the applications catch up and start using those cheaper commodity-level models instead. For now it’s easier to just throw an OpenAI API key into your application and let it use the heavyweight models for everything, a powerful model can do simple tasks just as well as a simple model. Most tasks are simple but adding the ability to distinguish those tasks from the complicated ones is hard.

Then they will go bankrupt, their assets and IP will be sold for pennies on the dollar, and those that follow them will be able to make a profit serving the established demand without the debt burden of the R&D that created it. It’s a common pattern for first-movers to not benefit from the industries they create.

That’s how it goes for any industry in its growth phase. A lot of money is spent on research and infrastructure before it starts to collect revenue.

You specifically referenced speed, though, which is emphatically not a legal drug. Amphetamines in general are tightly restricted.

Except doing drugs is illegal, whereas using AI is not. Fairly important distinction.

Need to make it sound apocalyptic somehow to draw the clicks.

How would bots be kept out of it?

Okay, he’s a leader of a large religion.

That still doesn’t give him any special knowledge or authority regarding AI.

Specifically sex abuse within the church.