Where Facebook's AI Slop Comes From - 0 views
-
For months, I have been documenting the incredible virality of bizarre AI-generated image spam on Facebook, now commonly referred to as “AI slop,” and Meta’s seeming complete apathy toward moderating this type of spam.
-
My investigation reveals that the AI images we see on Facebook are an evolution of a Facebook spam economy that has existed for years, driven by social media influencers, guides, services, and businesses in places like India, Pakistan, Indonesia, Thailand, and Vietnam, where the payouts generated by this content, which seems marginal by U.S. standards, goes further
-
The spam comes from a mix of people manually creating images on their phones using off-the-shelf tools like Microsoft’s AI Image Creator to larger operations that use automated software to spam the platform. I also know that their methods work because I used them to flood Facebook with AI slop myself as a test.
- ...8 more annotations...
-
Want to make images of giant Quarans and Bibles? There’s a guide for that. Optical illusion AI Jesus? Poor children? Poor people making intricate things out of plastic bottles? There are guides for them. Tricking people into clicking offsite? Avoiding bans? Getting an account unbanned? Posting automatically? There’s always a guide that explains every single phenomenon that I have seen while wading through AI-generated Facebook slop
-
These influencers are teaching people to use Facebook as a job. They are essentially penetration-testing Facebook, finding ever-changing vulnerabilities in its content moderation systems and in its recommendation algorithms and then exploiting them and instructing others how to do so at scale.
-
Meta does not make its payment rates public, even to people in the program. But YouTube influencers regularly show their payment dashboards. Payments for single images that I have seen vary wildly, from a few cents per photo to hundreds of dollars per photo if it goes megaviral. The “$100 for 1,000 likes” that influencer Gyan Abishek mentioned in one of his videos seems to be greatly exaggerated, based on the various payment portals that I have seen.
-
“If you can figure out how to post content at scale, that means you can figure out how to exploit weaknesses at scale,” the former Meta employee said.
-
. A Meta spokesperson told me that the company has 40,000 employees working globally on security and trust and safety today, compared to 20,000 in 2018.
-
The most popular way to make money spamming Facebook is by being paid directly by Facebook to do so via its Creator Bonus Program, which pays people who post viral content. This means that the viral “shrimp Jesus” AI and many of the bizarre things that have become a hallmark of Zombie Facebook have become popular because Meta is directly incentivizing people to post this content.
-
One former Meta employee with direct knowledge of its content moderation and ad approval systems, who spoke on the condition of anonymity because they signed an NDA at Meta, told me that Facebook is often aware of these loopholes but layoffs have left its content moderation teams so spread thin that they cannot actually keep up with how quickly people are exploiting them.
-
Much like similar programs at TikTok and Twitter, Facebook’s Creator Bonus Program makes direct payments to people who successfully go viral on Meta platforms, and is meant to incentivize influencers and content creators to post high-quality content on Facebook, Instagram, and Threads. Meta’s bonus program is “invite only,” but countless of the instructional videos I saw show that consistent posting over time will eventually get an account or page invited to the program.