Oh probably not, but it is really really expensive to run. If they continue on with unlimited funding, they can eventually replace mid to upper level experts in most fields.
We’ve likely hit the limit for the current LLM approach. Training/Pruning/Distilling can’t get us much further on their own, so now we’re strapping on code, having it plan, parallel execute, test. Tokens are likely 10x less expensive on the market than they cost to generate. Every advance we make now is by making each request fire off many more requests. Improvements are getting exponentially expensive.
We’ll have to slow because financially * it’s not profitable. Maybe hardware will catch back up. Maybe they’ll find ways to make it more efficient. Maybe we’ll figure out a more elegant way to do the math for the weights. We just can’t keep making it more deeply complicated because it’s not cost/energy/time effective
Oh probably not, but it is really really expensive to run. If they continue on with unlimited funding, they can eventually replace mid to upper level experts in most fields.
We’ve likely hit the limit for the current LLM approach. Training/Pruning/Distilling can’t get us much further on their own, so now we’re strapping on code, having it plan, parallel execute, test. Tokens are likely 10x less expensive on the market than they cost to generate. Every advance we make now is by making each request fire off many more requests. Improvements are getting exponentially expensive.
We’ll have to slow because financially * it’s not profitable. Maybe hardware will catch back up. Maybe they’ll find ways to make it more efficient. Maybe we’ll figure out a more elegant way to do the math for the weights. We just can’t keep making it more deeply complicated because it’s not cost/energy/time effective
edit: deleted a word