Every single one of us, as kids, learned the concept of “garbage in, garbage out”; most likely in terms of diet and food intake.
And yet every AI cultist makes the shocked pikachu face when they figure out that trying to improve your LLM by feeding it on data generated by literally the inferior LLM you’re trying to improve, is an exercise in diminishing returns and generational degradation in quality.
Why has the world gotten both “more intelligent” and yet fundamentally more stupid at the same time? Serious question.
Because the people with power funding this shit have pretty much zero overlap with the people making this tech. The investors saw a talking robot that aced school exams, could make images and videos and just assumed it meant we have artificial humans in the near future and like always, ruined another field by flooding it with money and corruption. These people only know the word “opportunity”, but don’t have the resources or willpower to research that “opportunity”.
Oh no. Anyways…
oh no are we gonna have to appreciate the art of human beings? ew. what if they want compensation‽
If mainstream blogs are writing about it, what would make someone think that AI companies haven’t thoroughly dissected the problem and are already working on filtering out AI fingerprints from the training data set? If they can make a sophisticated LLM, chances are they can find methods to XOR out generated content.
What would make me think that they haven’t “thoroughly dissected” it yet is that I’m a skeptic, and since I’m a skeptic I don’t immediately and without evidence believe that every industry is capable of identifying, dissecting, and solving every problem with its products.
Ironically given their skillset, training an ML model on known and properly tagged AI generated and non-AI-generated stuff might actually work.
Oh no
Anyway
Sooner or later it is supposed to happen, but I don’t think we are quite there…Yet.
Fake news, just like that one time Nightshade “killed” stable diffusion (literally had no effect) Flux came out not long ago and it’s better than ever
At this point the synthetic data is good enough to intentionally be used for training LLMs.
Yeah, just filter out the bad generated images and feed the good ones again, until the model learns how to produce only good ones.
is it not relatively trivial to pre-vet content before they train it? at least with aigen text it should be.
The problem is these AI companies currently exist on the business model of not paying for information, and that generally includes not wanting to pay content curators.
Google is probably the only one in a position to potentially outsource by making everyone solve a “does this hand look normal to you” CAPTCHA
They can try and train AI to detect AI, but that’s also difficult.
So it’s not a problem with AI. It’s just a problem for some mayfly companies that try to profit from the latest trend?
As always.
The model isn’t dying, its the way these parasites want it to work that is dying.
It depends on what you are looking for. Identifying AI generated data is generally hard, though it can be done in specific cases. There is no mathematical difference between the 1s and 0s that encoded AI generated data and any other data. Which is why these model collapse ideas are just fantasy. There is nothing magical about any data that makes it “poisonous” to AI. The kernel of truth behind these ideas is not likely to matter in practice.
Kind of like how true thoughts and opinions on complex topics are boiled down to digestible concepts for others to understand who then perpetuate those concepts without understanding them and the meaning degrades and we dont think anymore, just repeat stuff in social media comments.
Side note… this article sucks and seems like it was ai generated. Repetitive and no author credit? Just says it was originally posted elsewhere.
Generative AI isnt in danger of being killed as this clickbait titled suggests… just hindered.
hindered.
I doubt that.
By chance, is that based on other peoples succinct social media comments on ai?
No. I simply don’t see a plausible scenario for that. The social media comments are quite deplorable. You really have to look for bubbles with educated people. I don’t know why this gets so much traction. Maybe it’s because the copyright industry likes it, or maybe it feeds some psychological need like Intelligent Design.
Cant blame me for asking :)
Seems like tools to recognize ai content to prevent synthetic input avoids model degredation.
If those tools are up to the task then i would agree it probably doesnt hinder model training. Not sure what the reality is, or if the need for those tools creates a barrier to entry for a significant portion of those trying to create models with internet-crawled data.
There is no problem with ingesting synthetic data. Well, at least none coming from the fact that it is synthetic. If there was a fundamental difference between the 1s and 0s encoding synthetic data and the 1s and 0s encoding any other data, then you could easily filter it. But there isn’t. The ideas that this community has are magical thinking.
I want to be constructive so:
Please consider the unintentional disinformation people create when they try to sound like they know what they are talking about. Contributing to discussion is difficult on complex topics.
Its perfectly natural to want to continue a conversation to the point where you might fill in some details instead of researching a topic or not responding. But this is seriously harmful in the age of disinformation. Theres plenty i dont know. But there are tools expressly created to identify ai content to avoid using it in model training. The consequence of using synthetic data is the only topic in the article you are commenting on. Either read the article or please dont feel like you need to come up with a response.
Yes, I shouldn’t bother replying in these threads. In truth, I’ve already given up on this community but sometimes when I’m bored I can’t help a little peek. Maybe in a few years, some of the smarter ones will wonder why nothing ever came of this. Anyway, be careful with those AI detectors. They don’t work and sooner or later someone is going to get in trouble over that.
More like degenerative AIs
It is their own fault for poisoning the internet with their slop.
In case anyone doesn’t get what’s happening, imagine feeding an animal nothing but its own shit.
Photocopy of a photocopy is my go-to metaphor for model collapse.
I use the “Sistermother and me are gonna have a baby!” example personally, but I am a awful human so
Not shit, but isn’t that what brought about mad cow disease? Farmers were feeding cattle brain matter that had infected prions. Idk if it was cows eating cow brains or other animals though.
It was the remains of fish which we ground into powder and fed to other fish and sheep, whose remains we ground into powder and fed to other sheep and cows, whose remains we ground to powder and fed to other cows.
So yes. That’s what’s happening.
Maybe the internet will get a prion and die
Let’s go, already!
How you can help: If you run a website and can filter traffic by user agent, get a list of the known AI scrapers agent strings and selectively redirect their requests to pre-generated AI slop. Regular visitors will see the content and the LLM scraper bots will scrape their own slop and, hopefully, train on it.
This would ideally become standardized among web servers with an option to easily block various automated aggregators.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on - social media, public image storage, copyrighted media, etc. All those sites with extensive privacy policies who are signing contracts to permit their content for training.
Without laws (and I’m not sure I support anything in this regard yet), I do not see AI progress slowing. Clearly inbreeding AI models has a similar effect as in nature. Fortunately there is enough original digital content out there that this does not need to happen.
Regardless, all of us combined are a grain of rice compared to the real meat and potatoes AI trains on
Absolutely. It’s more a matter of principle for me. Kind of like the digital equivalent of leaving fake Amazon packages full of dog poo out front to make porch pirates have a bad day.
Well it means they need some ability to reject some content, which means they need a level of transparency they would never want otherwise.
They’ll just start using a chrome user agent
Only if enough people do it. Then again, loads scrapers outside of AI already pretend to be normal browsers.
You can validate that against user telemetry data expected from a browser.
AI already long ago stopped being trained on any old random stuff that came along off the web. Training data is carefully curated and processed these days. Much of it is synthetic, in fact.
These breathless articles about model collapse dooming AI are like discovering that the sun sets at night and declaring solar power to be doomed. The people working on this stuff know about it already and long ago worked around it.
I mean, we’ve seen already that AI companies are forced to be reactive when people exploit loopholes in their models or some unexpected behavior occurs. Not that they aren’t smart people, but these things are very hard to predict, and hard to fix once they go wrong.
Also, what do you mean by synthetic data? If it’s made by AI, that’s how collapse happens.
The problem with curated data is that you have to, well, curate it, and that’s hard to do at scale. No longer do we have a few decades’ worth of unpoisoned data to work with; the only way to guarantee training data isn’t from its own model is to make it yourself
Both can be true.
Preserved and curated datasets to train AI on, gathered before AI was mainstream. This has the disadvantage of being stuck in time, so-to-speak.
New datasets that will inevitably contain AI generated content, even with careful curation. So to take the other commenter’s analogy, it’s a shit sandwich that has some real ingredients, and doodoo smeared throughout.
They’re not both true, though. It’s actually perfectly fine for a new dataset to contain AI generated content. Especially when it’s mixed in with non-AI-generated content. It can even be better in some circumstances, that’s what “synthetic data” is all about.
The various experiments demonstrating model collapse have to go out of their way to make it happen, by deliberately recycling model outputs over and over without using any of the methods that real-world AI trainers use to ensure that it doesn’t happen. As I said, real-world AI trainers are actually quite knowledgeable about this stuff, model collapse isn’t some surprising new development that they’re helpless in the face of. It’s just another factor to include in the criteria for curating training data sets. It’s already a “solved” problem.
The reason these articles keep coming around is that there are a lot of people that don’t want it to be a solved problem, and love clicking on headlines that say it isn’t. I guess if it makes them feel better they can go ahead and keep doing that, but supposedly this is a technology community and I would expect there to be some interest in the underlying truth of the matter.
Old news? Seems to be a subject of several papers for some time now. Synthetic data has been used successfully already for very specific domains.
Yup, old news and wrong news. Also so many people who hate AI but don’t understand how it works. Pretty disappointing for a technology community.
…………………. Good?
Tbh I’m a bit lost on the purpose of this
“Model collapse” is just a fancy way of saying “our stupid ideas are bad and nobody wants them.”
No no. I think the LLMs. Or language models. Actually start to turn into mush “mentally” or how ever you phrase it.
