Inside the Booming 'AI Pimping' Industry (404media.co) 31
An anonymous reader quotes a report from 404 Media: Instagram is flooded with hundreds of AI-generated influencers who are stealing videos from real models and adult content creators, giving them AI-generated faces, and monetizing their bodies with links to dating sites, Patreon, OnlyFans competitors, and various AI apps. The practice, first reported by 404 Media in April, has since exploded in popularity, showing that Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
According to our review of more than 1,000 AI-generated Instagram accounts, Discord channels where the people who make this content share tips and discuss strategy, and several guides that explain how to make money by "AI pimping," it is now trivially easy to make these accounts and monetize them using an assortment of off-the-shelf AI tools and apps. Some of these apps are hosted on the Apple App and Google Play Stores. Our investigation shows that what was once a niche problem on the platform has industrialized in scale, and it shows what social media may become in the near future: a space where AI-generated content eclipses that of humans. [...]
Out of more than 1,000 AI-generated Instagram influencer accounts we reviewed, 100 included at least some deepfake content which took existing videos, usually from models and adult entertainment performers, and replaced their face with an AI-generated face to make those videos seem like new, original content consistent with the other AI-generated images and videos shared by the AI-generated influencer. The other 900 accounts shared images that in some cases were trained on real photographs and in some cases made to look like celebrities, but were entirely AI-generated, not edited photographs or videos. Out of those 100 accounts that shared deepfake or face-swapped videos, 60 self-identify as being AI-generated, writing in their bios that they are a "virtual model & influencer" or stating "all photos crafted with AI and apps." The other 40 do not include any disclaimer stating that they are AI-generated. Adult content creators like Elaina St James say they're now directly competing with these AI rip-off accounts that often use stolen content. Since the explosion of AI-generated influencer accounts on Instagram, St James said her "reach went down tremendously," from a typical 1 million to 5 million views a month to not surpassing a million in the last 10 months, and sometimes coming in under 500,000 views. While she said changes to Instagram's algorithm could also be at play, these AI-generated influencer accounts are "probably one of the reasons my views are going down," St James told 404 Media. "It's because I'm competing with something that's unnatural."
Alexios Mantzarlis, the director of the security, trust, and safety initiative at Cornell Tech and formerly principal of trust and safety intelligence at Google, started researching the problem to see where AI-generated content is taking social media and the internet. "It felt like a possible sign of what social media is going to look like in five years," said Mantzarlis. "Because this may be coming to other parts of the internet, not just the attractive-people niche on Instagram. This is probably a sign that it's going to be pretty bad."
According to our review of more than 1,000 AI-generated Instagram accounts, Discord channels where the people who make this content share tips and discuss strategy, and several guides that explain how to make money by "AI pimping," it is now trivially easy to make these accounts and monetize them using an assortment of off-the-shelf AI tools and apps. Some of these apps are hosted on the Apple App and Google Play Stores. Our investigation shows that what was once a niche problem on the platform has industrialized in scale, and it shows what social media may become in the near future: a space where AI-generated content eclipses that of humans. [...]
Out of more than 1,000 AI-generated Instagram influencer accounts we reviewed, 100 included at least some deepfake content which took existing videos, usually from models and adult entertainment performers, and replaced their face with an AI-generated face to make those videos seem like new, original content consistent with the other AI-generated images and videos shared by the AI-generated influencer. The other 900 accounts shared images that in some cases were trained on real photographs and in some cases made to look like celebrities, but were entirely AI-generated, not edited photographs or videos. Out of those 100 accounts that shared deepfake or face-swapped videos, 60 self-identify as being AI-generated, writing in their bios that they are a "virtual model & influencer" or stating "all photos crafted with AI and apps." The other 40 do not include any disclaimer stating that they are AI-generated. Adult content creators like Elaina St James say they're now directly competing with these AI rip-off accounts that often use stolen content. Since the explosion of AI-generated influencer accounts on Instagram, St James said her "reach went down tremendously," from a typical 1 million to 5 million views a month to not surpassing a million in the last 10 months, and sometimes coming in under 500,000 views. While she said changes to Instagram's algorithm could also be at play, these AI-generated influencer accounts are "probably one of the reasons my views are going down," St James told 404 Media. "It's because I'm competing with something that's unnatural."
Alexios Mantzarlis, the director of the security, trust, and safety initiative at Cornell Tech and formerly principal of trust and safety intelligence at Google, started researching the problem to see where AI-generated content is taking social media and the internet. "It felt like a possible sign of what social media is going to look like in five years," said Mantzarlis. "Because this may be coming to other parts of the internet, not just the attractive-people niche on Instagram. This is probably a sign that it's going to be pretty bad."
Who cares? (Score:5, Interesting)
The existence of human 'Influencers' is an embarrassment to humanity. If they're replaced by algorithms maybe they'll find something useful to do with their lives.
Re: (Score:3)
Re: (Score:2)
If we can train one to generate social media posts so I don't have anything to do on Slashdot anymore, I'll get my life back.
Re: Who cares? (Score:1)
The more this goes on, the more I realise that those of us who actually do work have nothing to fear from AI related job losses.
Re: (Score:2)
This is actually a bigger problem. We could all easily lounge about pretending to work, but what happens when all the pretend jobs are gone and those useless people need to be kept busy and docile?
Re: Who cares? (Score:1)
Same thing that happens now. They get addicted to the drugs that the government claims it's trying to get rid of and live homeless under a bridge.
Ruh-roh! (Score:2)
Re: (Score:2)
You crap on human "influencers" and yet I absolutely guarantee they influence you. There's some activity that you like where someone has influenced you in some way. But it is always cute to see those who believe they're not influenced by others. Like the idiots who claim that advertisements don't influence them, while science has shown otherwise countless times.
Re: (Score:2)
There's a difference between a random human influencing me (which happens all the time), and a "professional influencer" doing the same (mainly because I don't see them on account of skipping, moving on or doing literally anything to get away from such worthless rubbish content).
Don't generalise a specific discussion. If you want to know what we're talking about when using the term "influencer" look at TFA. If you are the type of person to watch the videos of those people in TFA, then please just set your m
There's a silver lining to this cloud (Score:2)
Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
So what they're saying is that thanks to AI, being an "influencer" might no longer be a viable career choice? I'm starting to like AI a little more every day!
Oh no! (Score:2)
Not cheap fake porn! That's horrid! I hope those monsters are shut down. I shall clutch my pearls immediately.
Re: (Score:2)
Not cheap fake porn! I shall clutch my pearls immediately.
You mean clench on your pearls don't you?
Dead Internet Theory (Score:5, Informative)
showing that Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
Meanwhile I've tried to create an Instagram account on multiple occasions and it gets suspended before I can even do anything with it. I've appealed, which requires presenting proof that you are an actual human, and they uphold it with some vague nonsense about "fake accounts" violating their policy (how they determine it's "fake" when I've proven I'm not a bot is beyond me).
So yeah ... they have an AI / fake account / bot problem on the platform? Meanwhile they're banning real humans who have no malintent what-so-ever and making it impossible for new users to sign up and even get an account started (I've searched and lots of other people complain about the same thing).
Maybe they're overrun with bot accounts because they've suspended all of the actual human users for no reason and all that's left are bots.
Re: Dead Internet Theory (Score:2)
That sounds like something a bot would say.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Meanwhile I've tried to create an Instagram account on multiple occasions and it gets suspended before I can even do anything with it.
What is interesting here is that you can't seem to do something that 2 billion other people have done. And among those people are influencers, who collectively struggle to share a single braincell.
I suspect you're doing something suspicious. Let me guess: signing up behind a VPN, using obviously fake emails, or maybe setting a swastika as your account logo?
Bot vs bots (Score:2)
Re: (Score:2)
It's mostly already there. It's why I deleted all of mine, amongst other reasons. Haven't missed it since 2017.
Re: (Score:2)
You mean the truth that Ukraine is losing, retreating on all fronts and will collapse shortly? Yeah, I keep up on that.
Re: (Score:2)
We have railroads.
Re: (Score:2)
Yeah I recently shut my twitter account for that reason. Between the P*SSY*IN*BIO* accounts and shitful political outrage bots constantly shouting american politics at me (I'm not american, I dont give a fuck about what y'all do over there we have our own problems to worry about, stop yelling it at me!) the place is lost. Facebook is currently crawling with identity theft bots. Instagram and TikTok, I have no idea, Instagram I dont see the appeal, and I am utterly mystified by Tiktok.
Bluesky seems nice, so
Re: (Score:2)
Hah. I still remember BBSing. I miss those people. The only reason I haven't dumped my FB account is a few old fogeys from the BBS days I only talk to on there.
My ex-wife was doing TikToks and they were so stupid...I figured out that I don't want to be there. I don't blame you.
The politics is what got me to delete all my accounts. Also, the realization that future employers would be checking our social media as part of their background checks. At that point, having an account sounded like idiocy.
Re: (Score:2)
Influencers forced to look for a real job (Score:2)
The horror!
Sad but true (Score:2)
So far the only uses for AI are crap generation and fraud
I'm hopeful that future AI will actually be useful, but today's AI is far from it