And it’s all because their models are super under-optimised.
You’d think otherwise with how capable Opus and Fable are, but it’s just because the internal harness hides the thinking process, resulting in wildly improved output at the cost of 3-5x more compute being burned.
If they tossed one of the more capable open models in (looking at you, Qwen3.8/Qwen4), those costs would go down a lot.
No, it’s not that these models are unoptimized. Infact the opposite. These models are optimised and are optimised well.
Gpt 6 Astra is a looped transformer. While idea sounds simple, loop the output through the transformer again and again, these transformers punch way above their weight in tasks that are iterative, like coding or higher level planning. Now again, by higher level planning I don’t mean your everyday tasks, it’s mostly the tasks that benefit from being revisited, actually more like refining.
I’ve read a 7b looped transformer beating qwen 72b coder model, and I’ll link it in the edit.
It’s naive to think these models are unoptimized, but it’s still even more naive to think these models are AGI. There are some genuinely great architectural advancements being made by both USA and China, yet I think it’s for the better this bubble pops as soon as possible. No technology in this world is worth destroying natural resources, and despite these advancements, LLMs are way too far from generating the economic output they project.
None of the LLMs could figure out what I meant when asked “Find me lion and the lamb iconography examples used before the messianic age”. They kept instead pulling up Christian things specifically (even though Judaism beats it by hundreds of years). Even when I emphasized to ignore Christianity / Christian sources they’d keep insisting on it.
It doesn’t matter how you process the data - looped transformer, harnesses, etc. These are still glorified overgrown statistics machines, and will default to whatever floods it’s data the most because there is ZERO intelligence.
I’ve also checked how they do code, and it’s terrible. The great for very beginner stuff or boiler plate code. By anything as advanced as a website and it’s shit.
Sure, it technically can make a functional website. I tested it just to see how it works. But it does it the equivalent way a human technically can make a functional car out of cobbled together scraps from a scrap yard. It used SO MANY <div> classes for no goddamn reason. So much inline CSS too. It’s like, good fucking luck fixing an issue manually and unravelling the tangle of containers it’s made. You’d have to use the LLM again just to change things at that point because it’d be easier, and hope it doesn’t break a button. Mind you this is a simple HTML and CSS website, not even JavaScript. There’s no way in hell I’d trust anything more complex created via vibe coding.
I definitely agree LLMs generate a lot of bloated code.
Look, LLMs are pattern identifiers. They identity patterns. They don’t guess. Guessing is what humans do. They ‘latch’ onto a pattern. They don’t learn patterns, they recognise them. Yes, is what studies have shown. LLMs don’t learn gradually like Humans. They “latch” onto patterns. It’s not sentient. It’s just a machine learning algorithm, and if you think any of the AI that came before LLM is intelligent, then even LLMs are intelligent. If you think deep learning algorithms are just non linear matrix multiplications, they’re exactly that.
LLMs coming for Coding jobs isn’t because they’re superhuman intelligent, its cause what most coders do is mostly same things. LLMs cannot do novel architecture, but if it’s similar to 50 different softwares who’s code it has trained on, yes, LLMs can so that.
LLMs are marvelous machines. They’re not sentient, and only intelligent if you’re willing to call every other ML algorithm intelligent. But the underlying statistics algorithm can and will keep getting better and/or more efficient, that was the point I was trying to make earlier.
Edit: actually you make some more good points. If LLM gets free reign over a project, they do make code unreadable. And Idk if that problem will be solved. It’s just too much code they write, and efficiency is not something their training data gave them.
And it’s all because their models are super under-optimised.
You’d think otherwise with how capable Opus and Fable are, but it’s just because the internal harness hides the thinking process, resulting in wildly improved output at the cost of 3-5x more compute being burned.
If they tossed one of the more capable open models in (looking at you, Qwen3.8/Qwen4), those costs would go down a lot.
No, it’s not that these models are unoptimized. Infact the opposite. These models are optimised and are optimised well.
Gpt 6 Astra is a looped transformer. While idea sounds simple, loop the output through the transformer again and again, these transformers punch way above their weight in tasks that are iterative, like coding or higher level planning. Now again, by higher level planning I don’t mean your everyday tasks, it’s mostly the tasks that benefit from being revisited, actually more like refining. I’ve read a 7b looped transformer beating qwen 72b coder model, and I’ll link it in the edit.
It’s naive to think these models are unoptimized, but it’s still even more naive to think these models are AGI. There are some genuinely great architectural advancements being made by both USA and China, yet I think it’s for the better this bubble pops as soon as possible. No technology in this world is worth destroying natural resources, and despite these advancements, LLMs are way too far from generating the economic output they project.
Edit: Multilingual-Multimodal-NLP/LoopCoder-V2 · Hugging Face https://huggingface.co/Multilingual-Multimodal-NLP/LoopCoder-V2
The version I was talking about was the 2 loop one.
None of the LLMs could figure out what I meant when asked “Find me lion and the lamb iconography examples used before the messianic age”. They kept instead pulling up Christian things specifically (even though Judaism beats it by hundreds of years). Even when I emphasized to ignore Christianity / Christian sources they’d keep insisting on it.
It doesn’t matter how you process the data - looped transformer, harnesses, etc. These are still glorified overgrown statistics machines, and will default to whatever floods it’s data the most because there is ZERO intelligence.
I’ve also checked how they do code, and it’s terrible. The great for very beginner stuff or boiler plate code. By anything as advanced as a website and it’s shit.
Sure, it technically can make a functional website. I tested it just to see how it works. But it does it the equivalent way a human technically can make a functional car out of cobbled together scraps from a scrap yard. It used SO MANY <div> classes for no goddamn reason. So much inline CSS too. It’s like, good fucking luck fixing an issue manually and unravelling the tangle of containers it’s made. You’d have to use the LLM again just to change things at that point because it’d be easier, and hope it doesn’t break a button. Mind you this is a simple HTML and CSS website, not even JavaScript. There’s no way in hell I’d trust anything more complex created via vibe coding.
I definitely agree LLMs generate a lot of bloated code.
Look, LLMs are pattern identifiers. They identity patterns. They don’t guess. Guessing is what humans do. They ‘latch’ onto a pattern. They don’t learn patterns, they recognise them. Yes, is what studies have shown. LLMs don’t learn gradually like Humans. They “latch” onto patterns. It’s not sentient. It’s just a machine learning algorithm, and if you think any of the AI that came before LLM is intelligent, then even LLMs are intelligent. If you think deep learning algorithms are just non linear matrix multiplications, they’re exactly that.
LLMs coming for Coding jobs isn’t because they’re superhuman intelligent, its cause what most coders do is mostly same things. LLMs cannot do novel architecture, but if it’s similar to 50 different softwares who’s code it has trained on, yes, LLMs can so that.
LLMs are marvelous machines. They’re not sentient, and only intelligent if you’re willing to call every other ML algorithm intelligent. But the underlying statistics algorithm can and will keep getting better and/or more efficient, that was the point I was trying to make earlier.
Edit: actually you make some more good points. If LLM gets free reign over a project, they do make code unreadable. And Idk if that problem will be solved. It’s just too much code they write, and efficiency is not something their training data gave them.
Anthropic raised $105B in 2026. $10.9B Q2 revenue. That’s at most $50B revenue per year. It’s still 50% burned. Don’t you fucking see it ?