I was thinking about this after a discussion at work about large language models (LLMs) - the initial scrape of the internet before Chat GPT become publicly usable was probably the last truly high quality scrape of human-made content any model will get. The second Chat GPT went public, the data pool became tainted with people publishing information from it. Future language models will have increasingly large percentages of their data tainted by AI-generated content, skewing the results away from how humans actually write. To get actual human content, they may need to turn to transcriptions of audio recordings or phone calls for training, and even that wouldn’t be quite correct because people write differently than they speak.
I sort of wonder if eventually people will start being influenced in how they choose to write based on seeing this AI content. If teachers use AI-generated texts in school lessons, especially at lower levels, will that effect how kids end up writing and formatting their work? It’s weird to think about the wider implications of how this AI stuff will ultimately impact society.
What’s your predictions? Is there a future where AI can get a clean, human-made scrape? Are we doomed to start writing like AIs?
It’s not going to replace actual dedicated writers, but it’s definitely going to hinder people learning to write and make up a large portion of the text online. It may also make it harder for actual writers to be found in all the noise. I heard a little while back about a scifi magazine which had to close its submissions because it was getting too many AI-written stories and sorting through the real versus fake was becoming difficult for them.
As for who’s going to train the AI, that’s part of what I’m arguing here - future LLMs are going to wind up being trained on AI-generated text because there will be so much of it online that screening it out becomes near impossible. Reddit mods already have challenges screening out chat GPT bots from their comments. When a future LLM scrapes the web for writen words, it’ll come back with lots of garbage AI text which will taint its learning pool. AIs will learn from AIs and become worse for it.
Yes but many industries, writing included, already suffer from fraudulent activity.
One of the largest reasons why entry-level software engineering jobs require five years experience is because consulting firms train developers (making the juniors) and have them interview as senior developers.
The firm manages the job search, helps the consultants during interviews, and they have teams helping with the actual work as well as fitting in with the rest of the, more experienced (or also fraudulent), staff.
There are currently industries which make a profit off of fooling other companies and consumers, which are arguably more frightening.
If anything, this will increase demand for better human writers and ways to authenticate their work. If it doesn’t, we’ll get sick of the content AI creates before it gets too bad.