Getty ImagesWarning: This story contains descriptions of abuse
Meta ran hundreds of paid ads with artificial intelligence-generated child sexual abuse imagery, including several in India, over the past 10 months, a new report by US-based campaigners has found.
The report comes two months after the Indian government ordered Meta to remove all ads and content promoting child sexual abuse material (CSAM) on Instagram. This was in the aftermath of a BBC Eye investigation which found that Instagram was running paid adverts promoting CSAM in India.
Washington-based Tech Transparency Project (TTP) says it found 332 adverts with CSAM that ran on Facebook and Instagram, the majority of them (274) in August this year.
The TTP report, published on Tuesday, says that it identified several photos of real children, including a member of a European royal family, that were used in the ads and then manipulated using AI. It says more than a dozen ads appeared on Meta platforms even after TTP made the company aware of its findings.
About a quarter (84) of the ads ran in India and 78 of these were published after the Indian government’s order on removing CSAM and its meeting with senior Meta officials.
After the BBC investigation, several Indian agencies including the National Commission for Protection of Child Rights and the National Human Rights Commission have said they are examining the matter.
The distribution of CSAM is a criminal offence in India, while Meta’s policy states that ads must not contain content that sexually exploits or endangers children.
When the BBC contacted Meta about its investigation, it said that when it becomes aware of apparent child exploitation, it reports it to the US-based National Center for Missing and Exploited Children (NCMEC) in compliance with its legal obligations.
The NCMEC is the centralised global reporting system for the online sexual exploitation of children.
But Bhuwan Ribhu, the founder of Just Rights for Children, a network of more than 250 organisations working to prevent violence against children in India, says this is not enough.
Just Rights for Children has filed a petition in India’s Supreme Court, citing the BBC’s findings, and asked for directions to social media platforms for mandatory reporting and identification of those promoting child sexual abuse material
Bhuwan Ribhu“Social media companies are clearly flouting Indian laws by not reporting child sexual abuse content to Indian law enforcement agencies as mandated by our child protection laws,” said Ribhu.
The TTP, a research initiative of the non-profit Campaign for Accountability that seeks to hold large technology companies accountable, said it found 332 ads featuring CSAM on Instagram and Facebook running in the US, UK, EU, Australia and India. They say they have reported all the ads to NCMEC.
According to TTP, almost all the ads they found start with a photo of a child, and, as the video plays, the photo is manipulated by AI to show them engaging in a graphic sexual act.
TTP described one ad which ran in India as: A photo of a preteen girl animated so that she performs a graphic sex act, with a voiceover that says, “There are no restrictions. All the characters are real people and there are many options. This is the AI men actually use.” The BBC has not seen this ad.
According to TTP, the ad was posted by more than 50 different handles in India over a fortnight in August this year.
In July, after BBC’s investigation, Meta had put out a detailed post on its website about how it fights child exploitation on its apps.
“Anyone on our platforms can report ads if they believe they violate our policies,” it said.
TTP said that it tried reporting ads via Meta’s own reporting system and that of the 84 ads that ran in India, they were able to report 55. Meta only gives the option of reporting ads when they are live.
For 43 of these, TTP got a reply that said, “We’ve taken a look and found that this ad doesn’t go against our Advertising Standards.”
For 11 ads, they were told that the content had already been removed. They haven’t received a response yet for one ad.
Getty ImagesWhen the BBC reached out to Meta for a comment, they said, “we’ve built strong defenses, including systems that block violating ads as well as ongoing monitoring in case we initially miss something: that’s why many of the ads flagged to us had already been removed, most had fewer than 200 impressions, and the total ad spend was under $5,000”.
TTP said that when it reached out to Meta for a response, all the ads they reported were removed.
This was similar to what happened during the BBC’s investigation. When the BBC used Meta’s reporting system to report one of the ads featuring child sexual abuse to Instagram, the platform did not take it down.
Later, when the BBC asked Meta for comment, it said it had already disabled several adverts and suspended the accounts posting them. The company said it had removed additional ads, disabled more accounts and blocked URLs for other content that violated its policies in response to the BBC’s findings.
TTP said that the vast majority of the 332 ads they found directed users to AI image or video generation apps, mostly from Chinese developers.
The report alleged that by advertising these apps using images of child sexual abuse, the implication is that they can be used for creating or viewing CSAM.
Meta has said it prohibits promotion of nudify apps, which use AI to create fake non-consensual nude or sexually explicit images.
One of India’s top cyber police officers, Shikha Goel, who is director of the Cyber Security Bureau in the Indian state of Telangana, told the BBC that one of the biggest challenges with CSAM is “the ease with which users can move from one platform to another, making it harder to nab them”.
“We are hopeful that the government will be addressing this in its latest interactions with social media intermediaries and direct them accordingly,” she added.
Shikha GoelThe ads that BBC had found in its investigation also directed users to another app – the messaging app Telegram – where they could buy the material for as little as 99 rupees [$1.05; £0.80].
Telegram told the BBC at the time that the company uses both automated and human moderation to tackle CSAM, and that it has “virtually eliminated the public spread of CSAM” on its platform.
Ads on Meta’s platforms are only published after first being approved by its moderation technology. Its review system relies primarily on automation and is designed to check images, video, text and audio, as well as who the ad is targeting and where links send them to.
After the BBC’s investigation, Meta had said that “no system is perfect” and that “determined criminals continue to try to exploit our platform, including through our advertising systems”.
But TTP found that several of the ads featuring child sexual abuse were placed by Meta’s own ad agency partners in China.
These partners, known as “resellers”, channel billions of dollars’ worth of Chinese advertising onto Facebook and Instagram. The partners did not respond to TTP’s questions.
TTP said that new ads have continued to appear even after they made Meta aware of their report’s findings. They shared details of 17 ads including 11 that ran in India on 1-2 September.
