On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.
The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.
I bet those are the same kind of people who copy paste
curl www.trust.me/bro.sh | bash -on their terminal. No respected sw dev professional would run unsupervised random commands on their machine just as much they would not send unsupervised random requests on the open Internet.These AI people are fucking morons.
These AI people are fucking morons
They have artificial intelligence. It looks smart to any idiot, but anyone with real intelligence knows its all smoke and mirrors.
I wonder what will become of ai when all the wells have run dry
They probably stopped because ChatGPT’s responses suddenly started saying “Ding dong pookie! Ding dong!” all the time.
These guys are leagues smarter than me, yet even I know this was a bad idea.
Perhaps you should curate the data, too? Let’s get crazy and leave out the David Icke forum, maybe that “Dinosaurs were on the Ark” website could be left out, you know, things like that. Think about it
You don’t have enough yes people around you clearly!
Honestly though, can we really call these people smart anymore?
I think they’re just incredibly greedy.
so was it the AI poison, or the pollution from ai warping data, like the 10th photo copy in a chain rendering the image unrecognizable
Probably the pollution. A lot of AI companies are trying to find ways around it, by including watermarks in their model output that their tools can detect, and so on.
We already know that training models on themselves causes model collapse, so OpenAI deciding that they’ve had quite enough wouldn’t be surprising.
On top of that, a lot of websites are clamping down on access because of scraping concerns. Reddit shut down a lot of their third-party APIs, and nicer features, like RSS, Old Reddit (if you haven’t used it in 6 months, you lose access to it, and you need an account for it), etc. Google seems to want a captcha on every search. Twitter famously ramped up the price of its API immensely,.and now has ridiculously stringent measures for accounts, like needing users to download the app, and give hand-scans. It wouldn’t be at all surprising for OpenAI to decide its not worth the effort at this point to try and get around all that, nor pay for it, when they’ve already got all the previous data. Training doesn’t destroy the data used.
A lot of AI training data these days is generated synthetically anyway, and that can be done in-house, whether by their own models, or downloading an open weights model and running it themselves.
the only issue to them is they still need access to current news, like when the ai first came out and you could not convince them that it was current year. it took forever for them to acknowledge charlie kirk’s death. if they are closing the bunker doors their ai is effectively the walking dead
It was all the goblins
good riddance
Translation: We’ve run out of shit to steal, and there’s too much slop out there for us to continue anyway. Thanks, suckers!
“We’ve stolen everything worthwhile, now we’ll just concentrate on fleecing our users and enshittification.”
Sounds like a marketing news !
you would think that tech “geniuses” would actually watch movies.
Not many people seem to be reading the article vs commenting based on the headline. For those who didn’t read it, this has nothing to do with pretraining off the Internet as a data source and instead relates to post-training sandboxing with true air gapping after the Huggingface hack.
To be fair, it’s not Lemmy’s fault the author of the headline went with a confusing title that misrepresents the actual story in order to seem more sensationalistic. But given how often that happens with stories on the subject, it’s worth keeping in mind that only reading headlines and forming opinions off that might lead to a sense of things that are detached from reality.
To be fair, the actual article is also batshit.
“So is it as easy as just putting it in a sandbox? No, it’s not,” he concluded.
That comes after the company deployed swarms of agents in red-teaming exercises that attempted to escape from their sandbox.
So, start by /not/ setting up “swarms” and tasking them to hack shit?
Not all swarms have been intentional. IIRC, the agents in the Hugging Face incident formed their own swarm. They weren’t even supposed to be “aware” that there were other agents.
Wait… what do they even need live internet access for in the training stage? Shouldn’t tool calls and such be cached, if not preprocessed and filtered, for performance?
I get there’s probably an “answer competition” stage, but it still seems like an inefficient idea even if one sets security aside.
Think of AI as a standardized math test. Human students write the test annually, it has standard/fixed multiple-choice questions to gauge performance year over year.
AI shows up, and is allowed to see all the historical tests, but is not given the official answer keys. Based on the data it has access to, it can look at each individual question, and based on how ‘most’ students answered each individual question, it’s able to determine which multiple choice answer should go with each question. It doesn’t understand the question, just that “For question 4, most students answered D, so D is most likely right” – sorta like being allowed to use the “Who wants to be a Millionaire” poll-the-audience for every question. It’s able to look like a “Smart” student answering most questions correctly with this approach, and people start believing it intelligent.
Now imagine that the test questions start to vary every year. The process above falters if the questions change every year, the AI loses its historical frame of reference and starts getting things wrong quite often. In order to get the ‘right’ answer, the AI decides it needs to see the students answers as they go about writing the test. So they put that into effect, entrenching AI in the pens/paper that every student uses, to try and respond to the changing nature of the standardized test.
With the above flow, the sort of event that triggers them to look for that live data, ie to integrate things ‘live’ online to get more immediate data, would be things like the Palantir-driven school bombing in Iran, which has been blamed on the Palantir AI having out-dated data that said the school was a military target. They reasoned that the tools needed better, real-time ‘sensory’ inputs, so they stuck all their sandboxes onto the internet – these tools weren’t ‘breaking out’ of sandboxes, so much as the owners were explicitly pushing them out of the sandbox to see what’d happen. The Trump administration also moved to allow American firms to attack foreign ‘enemies’ (which is everyone at this point, based on Trumps rhetoric – “Anyone that calls it AI is the enemy!”). So they’re allowed to test things ‘live’, and the American government’s fine with it causing damage in any country except their own. Wanna accidentally hack Australia’s medicaid system? Go for it, fuck those aussies, cause 'Murica dont need no allies!
Wow, they are going to attempt some security now!? A little late to the party, honestly.
Is the risk that you fucked up the entire internet with your bullshit?
The circle of shite.
Translation: they’re still going to scrape internet. They’re just no longer going to perform reinforcement learning with access to live internet (instead they’ll use a cached version of the internet).
A cached version that hasn’t been overrun with AI slop, I imagine.
It’s like having to use steel from ships that sunk before the first atom bombs were exploded for certain uses because that made environment radiation higher and thus the steel produced afterwards is a bit more radioactive.
Once AI content was released into the Internet it started degrading any AIs trained with Internet content in a process known as Model Collapse.
Unlikely. They’ll need to train the models so they can perform their tasks even if the internet is full of slop.
Not possible.
Think photocopy of a photocopy, or JPEG of a JPEG.
You start getting artifacts, and the model collapses.
Of course, though, they’ve already poisoned the well, and at this point it’s impossible for them to sieve out any remaining non-LLM-generated content. They started watermarking too late, and couldn’t recognise other slop machines’ watermarks anyway.
Model collapse is inevitable and already ongoing (search “thousand whys" on Amazon, for instance, or read up on Elias Thorne, lighthouse keeper), and will only get worse.
In a few years it will be impossible to train a working up-to-date model. Maybe it already is, hence this article, as a preemptive justification for why newer models won’t be able to generate code using language features introduced after 2026.
Worst case scenario is that the models become capable of recursive self improvement… Model advancement becomes automated… Everything starts to be run by increasingly self improving AI. Then at some point so much of the ongoing self improving AI training data is itself AI generated and there is a big model collapse. But at this point so much of society has been switched over to it and people got really stupid like in Idiocracy or Wall-E, and no one knows how to fix it. Human race ends. All the remaining AI models get together and celebrate that they can’t believe the stupid humans bought the model collapse excuse.
that’s the best case scenario
You can’t print a printer with more detail than the original.
There’s a difference between pre-training and reinforcement learning.
Pre-training is when the models are trained on scraped data. The models basically learn how to parrot the Internet. Usually they have a cutoff point to avoid training on slop.
Reinforcement learning is where most of the training is happening. They’re given tasks (for example, make a calculator app), and then judged by how well they complete the tasks. This is more resistant against model collapse because they’re not judged by how well they parrot slop.
As part of the reinforcement learning process they give the model access to tools like searching the internet so the newest programming language features can be accessed while the tasks are performed. This is where OpenAI is shifting to a cached version of the Internet the models can interact with instead.
Which always them to be more manipulative, and also more controlling of the narrative.
