Inside the Booming 'AI Pimping' Industry (404media.co) 101
An anonymous reader quotes a report from 404 Media: Instagram is flooded with hundreds of AI-generated influencers who are stealing videos from real models and adult content creators, giving them AI-generated faces, and monetizing their bodies with links to dating sites, Patreon, OnlyFans competitors, and various AI apps. The practice, first reported by 404 Media in April, has since exploded in popularity, showing that Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
According to our review of more than 1,000 AI-generated Instagram accounts, Discord channels where the people who make this content share tips and discuss strategy, and several guides that explain how to make money by "AI pimping," it is now trivially easy to make these accounts and monetize them using an assortment of off-the-shelf AI tools and apps. Some of these apps are hosted on the Apple App and Google Play Stores. Our investigation shows that what was once a niche problem on the platform has industrialized in scale, and it shows what social media may become in the near future: a space where AI-generated content eclipses that of humans. [...]
Out of more than 1,000 AI-generated Instagram influencer accounts we reviewed, 100 included at least some deepfake content which took existing videos, usually from models and adult entertainment performers, and replaced their face with an AI-generated face to make those videos seem like new, original content consistent with the other AI-generated images and videos shared by the AI-generated influencer. The other 900 accounts shared images that in some cases were trained on real photographs and in some cases made to look like celebrities, but were entirely AI-generated, not edited photographs or videos. Out of those 100 accounts that shared deepfake or face-swapped videos, 60 self-identify as being AI-generated, writing in their bios that they are a "virtual model & influencer" or stating "all photos crafted with AI and apps." The other 40 do not include any disclaimer stating that they are AI-generated. Adult content creators like Elaina St James say they're now directly competing with these AI rip-off accounts that often use stolen content. Since the explosion of AI-generated influencer accounts on Instagram, St James said her "reach went down tremendously," from a typical 1 million to 5 million views a month to not surpassing a million in the last 10 months, and sometimes coming in under 500,000 views. While she said changes to Instagram's algorithm could also be at play, these AI-generated influencer accounts are "probably one of the reasons my views are going down," St James told 404 Media. "It's because I'm competing with something that's unnatural."
Alexios Mantzarlis, the director of the security, trust, and safety initiative at Cornell Tech and formerly principal of trust and safety intelligence at Google, started researching the problem to see where AI-generated content is taking social media and the internet. "It felt like a possible sign of what social media is going to look like in five years," said Mantzarlis. "Because this may be coming to other parts of the internet, not just the attractive-people niche on Instagram. This is probably a sign that it's going to be pretty bad."
According to our review of more than 1,000 AI-generated Instagram accounts, Discord channels where the people who make this content share tips and discuss strategy, and several guides that explain how to make money by "AI pimping," it is now trivially easy to make these accounts and monetize them using an assortment of off-the-shelf AI tools and apps. Some of these apps are hosted on the Apple App and Google Play Stores. Our investigation shows that what was once a niche problem on the platform has industrialized in scale, and it shows what social media may become in the near future: a space where AI-generated content eclipses that of humans. [...]
Out of more than 1,000 AI-generated Instagram influencer accounts we reviewed, 100 included at least some deepfake content which took existing videos, usually from models and adult entertainment performers, and replaced their face with an AI-generated face to make those videos seem like new, original content consistent with the other AI-generated images and videos shared by the AI-generated influencer. The other 900 accounts shared images that in some cases were trained on real photographs and in some cases made to look like celebrities, but were entirely AI-generated, not edited photographs or videos. Out of those 100 accounts that shared deepfake or face-swapped videos, 60 self-identify as being AI-generated, writing in their bios that they are a "virtual model & influencer" or stating "all photos crafted with AI and apps." The other 40 do not include any disclaimer stating that they are AI-generated. Adult content creators like Elaina St James say they're now directly competing with these AI rip-off accounts that often use stolen content. Since the explosion of AI-generated influencer accounts on Instagram, St James said her "reach went down tremendously," from a typical 1 million to 5 million views a month to not surpassing a million in the last 10 months, and sometimes coming in under 500,000 views. While she said changes to Instagram's algorithm could also be at play, these AI-generated influencer accounts are "probably one of the reasons my views are going down," St James told 404 Media. "It's because I'm competing with something that's unnatural."
Alexios Mantzarlis, the director of the security, trust, and safety initiative at Cornell Tech and formerly principal of trust and safety intelligence at Google, started researching the problem to see where AI-generated content is taking social media and the internet. "It felt like a possible sign of what social media is going to look like in five years," said Mantzarlis. "Because this may be coming to other parts of the internet, not just the attractive-people niche on Instagram. This is probably a sign that it's going to be pretty bad."
Who cares? (Score:5, Interesting)
The existence of human 'Influencers' is an embarrassment to humanity. If they're replaced by algorithms maybe they'll find something useful to do with their lives.
Re:Who cares? (Score:5, Funny)
Re: (Score:3)
If we can train one to generate social media posts so I don't have anything to do on Slashdot anymore, I'll get my life back.
Re: Who cares? (Score:5, Funny)
The more this goes on, the more I realise that those of us who actually do work have nothing to fear from AI related job losses.
Re: (Score:2)
This is actually a bigger problem. We could all easily lounge about pretending to work, but what happens when all the pretend jobs are gone and those useless people need to be kept busy and docile?
Re: Who cares? (Score:1)
Same thing that happens now. They get addicted to the drugs that the government claims it's trying to get rid of and live homeless under a bridge.
Re: (Score:2)
That's another, different, failure of society. There will always be outliers, but we have too many people without hope of social mobility.
If 'influencer' is your desperation gig, then I'd say that should be a problem too. Every time I've become aware of an influencer, they got there pursuing easy money. Of course, to actually succeed for a sustained period of time it takes effort. That still doesn't make it a worthy job yo me - it shouldn't exist as an option and we shouldn't have people desperate enoug
Re: (Score:2)
There are levels and levels to what you said. For those of us with skills, automation is less of a threat to us, but doing physical labor, or even lower tier positions like stocking shelves should be considered to be real work. AI and automation is going to be a threat to those working purely physical jobs that do not require a lot of thought, just because those jobs can be more easily replaced by robots. This has been the trend for hundreds of years, every major technological jump has caused a loss
Re: Who cares? (Score:1)
That's been a thing ever since we figured out that an ox can do the work of 5 men.
Ruh-roh! (Score:2)
Re: (Score:2)
The drawback is now the streets will be flooded with out of work "content creators" who continually run up to strangers to demonstrate how to do makeup blending.
Re: (Score:3)
There's a difference between a random human influencing me (which happens all the time), and a "professional influencer" doing the same (mainly because I don't see them on account of skipping, moving on or doing literally anything to get away from such worthless rubbish content).
Don't generalise a specific discussion. If you want to know what we're talking about when using the term "influencer" look at TFA. If you are the type of person to watch the videos of those people in TFA, then please just set your m
Re: (Score:1)
This is statistical illiteracy at work. Studies in question determine if statistically relevant groups change behavior meaningfully.
It's almost impossible to prove how much specific action by a specific person altered specific individual's actions. So we don't, because on population level individuals that aren't extreme outliers are irrelevant. (Technically some do, but those studies rarely if ever replicate).
And extreme outliers is what error margins are for.
Re: (Score:2)
Re: (Score:2)
Happy to provide some studies. Let's be real, you'll just deny it though.
https://onlinelibrary.wiley.co... [wiley.com]
https://journals.indexcopernic... [indexcopernicus.com]
https://www.psu.edu/news/resea... [psu.edu]
https://www.forbes.com/sites/w... [forbes.com]
https://pmc.ncbi.nlm.nih.gov/a... [nih.gov]
https://famesters.com/blog/how... [famesters.com]
Re: (Score:2)
Re: (Score:2)
It shouldn't be allowed!!!"
Now the "AI" is eating their lunch. I won't shed a tear.
Here, FTFY.
Re: (Score:3, Interesting)
The arts have always been shit on by philistines. Just because you don't get anything from it, doesn't mean it has no value to other people.
We had the same thing when popular authors started to emerge, being criticized for not doing something productive like praising god or labouring. Then again when professional sports became a thing. These days it's often humanities degrees.
Re: (Score:2)
If influencers are artists then our culture is truly bankrupt.
Re: (Score:3)
Again, it's always been the case that most artists are not Picasso.
Re: (Score:1)
The existence of human 'Influencers' is an embarrassment to humanity.
You complaint suggests this embarrassment is somehow a new phenomenon. Ever wonder what they called ‘influencers’ for the last few hundred years before someone decided to re-brand them for the internet age? Let’s stop pretending marketing doesn’t know how to re-brand shit for marketing and fucks sake.
If marketing expenses were not on the “right” side of corporate tax deductions, all of Capitalism would see 90% of marketing spend as a complete failure and proven financia
There's a silver lining to this cloud (Score:3)
Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
So what they're saying is that thanks to AI, being an "influencer" might no longer be a viable career choice? I'm starting to like AI a little more every day!
Re: (Score:2)
Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
So what they're saying is that thanks to AI, being an "influencer" might no longer be a viable career choice? I'm starting to like AI a little more every day!
Yep, we've finally found a use for AI.
Oh no! (Score:2)
Not cheap fake porn! That's horrid! I hope those monsters are shut down. I shall clutch my pearls immediately.
Re: (Score:3)
Not cheap fake porn! I shall clutch my pearls immediately.
You mean clench on your pearls don't you?
Re: (Score:2)
I think there are wider implications. I imagine that real people will soon not be able to compete with artificially generated "perfect" body images, just like no-one would want to see me in one of those videos in comparison to a human model today.
It's harmful because once again it will hold people, especially young people and fox news audiences, up to literally unreachable standards.
Both men and women will be hurt by this.
Re: (Score:2)
They already do give off unreal image, these people put on a fake person to sell an image. At least with AI generated influencers we will know they are a lie.
And its not just about the perfect body image, its look at my wonderful life etc.
Maybe I am wrong I don't watch any influencers personally.
Dead Internet Theory (Score:5, Informative)
showing that Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
Meanwhile I've tried to create an Instagram account on multiple occasions and it gets suspended before I can even do anything with it. I've appealed, which requires presenting proof that you are an actual human, and they uphold it with some vague nonsense about "fake accounts" violating their policy (how they determine it's "fake" when I've proven I'm not a bot is beyond me).
So yeah ... they have an AI / fake account / bot problem on the platform? Meanwhile they're banning real humans who have no malintent what-so-ever and making it impossible for new users to sign up and even get an account started (I've searched and lots of other people complain about the same thing).
Maybe they're overrun with bot accounts because they've suspended all of the actual human users for no reason and all that's left are bots.
Re: Dead Internet Theory (Score:5, Funny)
That sounds like something a bot would say.
Re: (Score:2)
Deserved funny, but I think YouTube is even worse. But the OP didn't mention it. Actually that makes me think Facebook and the cesspool formerly known as Twitter also deserve mentions in this context. (And LinkedIn has been nagging me to verify myself for a long time via a process that doesn't seem to apply to me. Stupid humans!)
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Re: (Score:1)
Meanwhile I've tried to create an Instagram account on multiple occasions and it gets suspended before I can even do anything with it.
What is interesting here is that you can't seem to do something that 2 billion other people have done. And among those people are influencers, who collectively struggle to share a single braincell.
I suspect you're doing something suspicious. Let me guess: signing up behind a VPN, using obviously fake emails, or maybe setting a swastika as your account logo?
Re: (Score:3)
Meanwhile I've tried to create an Instagram account on multiple occasions and it gets suspended before I can even do anything with it. I've appealed, which requires presenting proof that you are an actual human, and they uphold it with some vague nonsense about "fake accounts" violating their policy.
They've been doing you a great favor.
Re: (Score:2)
Yep. My one and only attempt to sign up on Instagram died on the vine last week. Almost immediately suspended, for no reason I can imagine. And so ended my desire to bother with it.
It offered me an option to appeal, and a deadline... but since I was quite pleased to be denied, I did not pursue it.
Re: (Score:2)
I had the same problem a few months ago trying to create a Facebook account. My account was immediately suspended as I created it. Their loss I guess!
Re: (Score:2)
I was looking into this recently on Facebook, and I assume its the same with Insta, and it seems that the real problem is trying to create an account using a computer instead of a mobile phone. They never approve an account created made with a computer. But you can use a non-provisioned phone to download the Facebook Lite app over wifi and easily create an account without having to prove anything and without giving them any real personal data. Once that is done, you can access the account with a computer
Bot vs bots (Score:3, Insightful)
Re: (Score:3)
It's mostly already there. It's why I deleted all of mine, amongst other reasons. Haven't missed it since 2017.
Re: (Score:2)
We have railroads.
Re: (Score:2)
You mean the truth that Ukraine is losing, retreating on all fronts and will collapse shortly? Yeah, I keep up on that.
And by retreating on all fronts you mean having advanced hundreds of square miles into Russia and holding that territory, and advancing further of late while simultaneously hitting ammo dumps, oil refineries, and C&C. Meanwhile, the Muscovite midget has to beg North Korea for ammunition and troops because he's "winning" by losing 1,500 troops every single day (on average).
Re: (Score:1)
The interesting part here is that you're both correct, and don't realize it.
Because you're thinking from our normative perspective. "But you're losing so many people for so few gains, no sane nation by our standard would view that as a victorious war".
While he at least partially understands the culture outside our European Christian bubble post foundational event that was 30 Years War, where we consider all people to have a spark of Holy Spirit within them and therefore are inherently valuable. Including ou
Re: (Score:1)
This is the focus on human life that I'm referring to. You like me and the poster above find human life to be the ultimate value around which everything else revolves.
Russians don't. They find their imperial project to be of ultimate value. Sacrificing lives domestically doesn't serve it, sacrificing them to take Ukraine (and probably other nations/territories in Eastern Europe) does.
This is the value disconnect that renders understanding borderline impossible for many people.
Re: (Score:2)
As a follow up, forgot to mention Ukraine eviscerating Russia's Black Sea Fleet by destroying one third of it despite not having a large naval force of its own. In fact, Russia's navy has been so humiliatingly defeated, it's had to relocate its remaining ships to Russian ports for fear of them being sunk. And this includes sinking a Russian submarine.
Re: (Score:1)
You really mean the US destroying a few ships in the Black Sea, the Ukrainians lack the capability. Do you really think that is significant? The Russians still control the sea and there was no real naval action anyway.
This is what I mean about delusion. You probably think six missiles fired into Russia, with five being destroyed and one damaged, is a successful attack, also. While several square kilometers of Ukraine are evacuated daily and handed over to the Russians and the military fraction of the co
Re: (Score:1)
Prime example of how Russian propaganda is making its way to the US
Re: (Score:2)
Truth as propaganda is the problem with you indoctrinated people. You don't know the truth but are convinced by the neocon propaganda. Same as the idiots who cheered the Patriot Act back in the early 2000s, and the Iraq invasion. At one time that was popular, and same suspects producing the same bullshit propaganda you are swallowing now.
Re: (Score:2)
It's a little difficult to understand your ramblings, but it seems you're suggesting that Russia's invasion of Ukraine is going as well as the US's invasion of Iraq. Not entirely untrue, the big difference is Russia wasn't able to get air or ground superiority and now Ukraine is even striking inside Russia. Still both ended up being long and protracted military operations
Re: (Score:2)
Unless Gallup is also run by Putin-bots, this thing is over anyway. [gallup.com]
Game, set, match.
Re: (Score:2)
Half the country is war-weary means Russia's 3-day operation is going well? You must be playing a completely different game than the rest of us.
It's been a disaster of an invasion for Russia. They're throwing hundreds of thousands of bodies to get small gains
Re: (Score:2)
Yeah I recently shut my twitter account for that reason. Between the P*SSY*IN*BIO* accounts and shitful political outrage bots constantly shouting american politics at me (I'm not american, I dont give a fuck about what y'all do over there we have our own problems to worry about, stop yelling it at me!) the place is lost. Facebook is currently crawling with identity theft bots. Instagram and TikTok, I have no idea, Instagram I dont see the appeal, and I am utterly mystified by Tiktok.
Bluesky seems nice, so
Re: (Score:2)
Hah. I still remember BBSing. I miss those people. The only reason I haven't dumped my FB account is a few old fogeys from the BBS days I only talk to on there.
My ex-wife was doing TikToks and they were so stupid...I figured out that I don't want to be there. I don't blame you.
The politics is what got me to delete all my accounts. Also, the realization that future employers would be checking our social media as part of their background checks. At that point, having an account sounded like idiocy.
Re: (Score:2)
You mean like people spot you as an obsessive mental case? No.
Get a life and get over your tears about Ukraine falling into Russian hands.
Re: (Score:2)
I'm paid by the US government. I'm just not an indoctrinated idiot. You'd be surprised how many of us there are withiin the organization. Picking out the winners and losers in warfare is pretty much our jobs, and Ukraine is a loser.
Re: (Score:1)
Yeah, Bluesky just has two problems, the CSAM and no chuds to own so the fanatics are starting to turn on each other. Which is the inevitable result of unending purity tests.
Re: (Score:2)
Sounds good to me, it will end up being just another huge waste of resources in an effort to transfer wealth from one person to another.
Re: (Score:2, Funny)
Influencers forced to look for a real job (Score:3)
The horror!
Sad but true (Score:1)
So far the only uses for AI are crap generation and fraud
I'm hopeful that future AI will actually be useful, but today's AI is far from it
It's all fantasy anyway... (Score:3)
People who follow influencers are just looking through a doorway into a fantasy world. Dreaming of being rich and beautiful, or whatever. It's a form of escapism: in real life, you are a Starbucks barista, but wouldn't it be great if you could sail around on your private yacht? If AI can present better fantasies than real people running the faces (and lives) through filters? Go for it.
Personally, my escape is reading books. I would much rather read a purely fictional book than a pimped-up autobiography. Continuing the book analogy: note that there are real people behind the fake influencer, scripting what will be shown and what narrative will be told. They are the authors of the fantasy.
The people feeling insulted here are the influencers who take themselves too seriously to begin with. They may not actually understand that they are just selling fantasies and escapism.
Re: (Score:3)
Intelligent people discuss ideas.
Average people discuss events.
Stupid people discuss other people.
You seem to be the former, being attracted by ideas (reading books, which present ideas).
Those who are attracted by autobiographies are (with very few exceptions) part of the latter.
Re: (Score:2)
If AI can present better fantasies than real people running the faces (and lives) through filters? Go for it.
that would be regular competition. what these influencers supposedly claim is that their content is being reused with minor ai-generated modification. they should be crying good old "bloody copyright infringement murder" but "ai-pimping" i guess gets more clicks these days.
if the stupid language weren't giveaway enough, mind the source. the whole story sounds exaggerated and could be an outright fabrication, maybe even ai-generated aswell. they claim to have "reviewed more than 1000 AI-generated Instagram a
My heart bleeds for them (Score:1)
Reminds me of some Slashdotter's sig I used to see: "Boycott shampoo! Demand real poo!"
So there is hope (Score:2)
"and it shows what social media may become in the near future: a space where AI-generated content eclipses that of humans"
So there is hope after all :-P
Something to consider (Score:3, Informative)
Is it in Instragram's interest to prevent AI from taking over niches it can take over?
Instagram has basically demonstrated that with is business model of maximizing female intra-sexual competition that utterly blows up women's competitive patterns driving them to insanity in record numbers that the only thing it cares about is maximizing engagement.
AI will certainly increase engagement, as it will optimize for attention seeking and female beauty far better than overwhelming majority of humans with a team behind them can.
I suspect the only thing where Instagram won't want AI to take over, is the point where punters, aka thirsty men and envious women will stop considering that they have a chance to score with the model for men, and get correct advice on plastic surgery needed to look similar to model for women.
porn was already about selling something fake (Score:2)
Perhaps Ms St James (obviously her real name) is finding her industry low hanging fruit for AI "replacement" because what she was selling was already fake on so many levels in the first place (despite being organic...ok well augmented organics)?
Re: (Score:2)
Not surprising since adult entertainment (porn and online gambling) is usually the first to adopt new tech. I have very little remorse for the "influencers" being displaced since many (probably most) are really just outsourced salespeople showing off "amazing" things for you to buy.
Re: (Score:2)
I'm sure it was nice getting money for being a product pedestal, but I don't see the need to put the pedestal on a pedestal.
Re: (Score:2)
The "real" people aren't real either (Score:4, Insightful)
Even before the AI explosion, there was so much fake going on that calling the source material "real" is already a stretch. For decades, fashion has shifted from natural beauty to fake beauty anyway. For all we know the source material was rendered in a 3D engine. At the very least, they are the result of plastic surgery, carefully crafted camera angles, and tons of filters.
Agricultural work (Score:2)
Meanwhile, at least here in the US, we have a shortage of people willing to pick produce. That gap is largely filled by illegal immigrants. Any "influencer" who is out of a job is welcome to come here to Florida and pick strawberries. The local
Porn quality (Score:3, Insightful)
This just opens the door for bespoke porn performers to offer something that that the mass ersatz porn lacks. A little personal touch if you will.
Elaina St James (Score:2)
Would.
[What was that, dear? I'm just posting on a technology web site.]