Thousands of images of child sexual abuse are hidden at the core of popular AI image generators, according to a new report that calls on companies to take action to fix a harmful flaw in the technology they produce.
The same images have made it easy for AI systems to produce realistic and explicit images of fake children and turn social media photos of fully clothed real teenagers into nude photos; This alarmed schools and law enforcement around the world.
Until recently, anti-exploitation researchers thought the only way some uncontrolled AI tools could produce exploitative images of children was by combining what they learned from essentially two separate sets of online images: adult pornography and benign children’s photos.
But the Stanford Internet Observatory found more than 3,200 images of suspected child sexual abuse in the giant AI database LAION, an index of online images and captions used to train leading AI image generators like Stable Diffusion. The watchdog group, based at Stanford University, worked with the Canadian Center for Child Protection and other anti-exploitation charities to identify illegal material and report original photo links to law enforcement.
The answer came immediately. On the eve of the Stanford Internet Observatory’s report being released Wednesday, LAION told The Associated Press that it had temporarily removed the datasets.
LAION, which represents the nonprofit Large-Scale Artificial Intelligence Open Network, said in a statement that it “has a zero-tolerance policy against illegal content and, out of an abundance of caution, we have previously removed LAION datasets to ensure they are safe.” We are republishing them.”
Although the images make up only a fraction of LAION’s index of nearly 5.8 billion images, the Stanford group says this likely affects the ability of AI tools to produce harmful output, reinforcing previous exploitation of real victims who appear multiple times.
David Thiel, chief technologist at the Stanford Internet Observatory, who authored the report, said this is not an easy problem to fix and that many productive AI projects rely on “effectively bringing them to market” and making them widely accessible because the field is so competitive.
“Doing an internet-wide crawl and preparing that dataset to train models is something that should be limited to a research operation and not something that should be open sourced without much more stringent attention,” Thiel said. he said in an interview.
One of the leading LAION users who helped shape the development of the dataset is London-based startup Stability AI, maker of the Stable Diffusion text-to-image models. New versions of Stable Diffusion have made it much more difficult to create malicious content, but the older version introduced last year (which Stability AI says it has not released) is still included in other apps and tools and remains “the most popular model for creation,” according to the Stanford report. clear images”.
“We can’t undo this. This model is in the hands of too many people on their domestic machines,” said Lloyd Richardson, director of information technology at the Canadian Child Protection Centre, which operates Canada’s hotline for reporting online sexual abuse.
On Wednesday, Stability AI said it hosts only filtered versions of Stable Diffusion and that it has “taken proactive steps to reduce the risk of misuse of Stability AI since taking over the proprietary development of Stable Diffusion.”
“These filters prevent unsafe content from reaching models,” the company said in a prepared statement. “By removing this content before it reaches the model, we can help prevent the model from producing unsafe content.”
LAION was the brainchild of German researcher and teacher Christoph Schuhmann, who told the AP earlier this year that one reason for making such a large image database publicly available was to ensure that the future of AI development would not be controlled by one authority. a handful of powerful companies.
“It will be much safer and much fairer if we can democratize this so that the entire research community and the general public can benefit from it,” he said.
Most of LAION’s data comes from another source called Common Crawl, a constantly crawled data repository from the open internet, but Rich Skrenta, Common Crawl’s general manager, said it’s LAION’s “duty” to crawl and filter what’s needed before using it. from him.
LAION said this week that it has developed “rigorous filters” to detect and remove illegal content before publishing datasets and is still working to improve those filters. The Stanford report acknowledged that LAION developers made some attempts to filter out “underage” explicit content, but that they could have done a better job if they had consulted with child safety experts first.
Most text-to-image generators are derived in some way from the LAION database, but it is not always clear which ones. OpenAI, the creator of DALL-E and ChatGPT, said it does not use LAION and has tweaked its models to reject requests for sexual content involving minors.
Google built the text-to-image Imagen model based on a LAION dataset, but decided not to make it public after an audit of the database in 2022 “uncovered a wide range of inappropriate content, including pornographic images, racial slurs, and harmful social stereotypes.”
Trying to retroactively scrub data is difficult, so the Stanford Internet Observatory is calling for tougher measures. One of these is for anyone who creates training sets from LAION-5B (named for the more than 5 billion image-text pairs it contains) to “delete them or work with intermediaries to clean up the material.” Another is to effectively eliminate an old version of Stable Diffusion except in the darkest corners of the internet.
“Legitimate platforms may stop offering versions of this for download, especially if they are frequently used to create malicious images and there are no security measures in place to block them,” Thiel said.
As an example, Thiel put forward the CivitAI platform, which is preferred by people who make pornography created by artificial intelligence, but said it does not have security measures against creating images of children. The report also calls on Hugging Face, the artificial intelligence company that distributes training data for models, to implement better methods for reporting and removing links to malicious material.
Hugging Face said it regularly works with regulators and child safety groups to identify and remove abusive material. CivitAI did not respond to requests for comment sent to its website.
The Stanford report also questions whether any photos of children — even the most innocuous ones — should be fed into AI systems without their parents’ permission, because of protections in the federal Children’s Online Privacy Protection Act.
Rebecca Portnoff, director of data science at anti-child sexual abuse organization Thorn, said her organization has conducted research showing that the prevalence of AI-generated images among abusers is small but steadily increasing.
Developers can mitigate these harms by ensuring that the datasets they use to develop AI models are cleansed of abusive material. There are also opportunities to reduce harmful uses once the models are already in circulation, Portnoff said.
Tech companies and child safety groups currently assign a “hash” (unique digital signatures) to videos and images to track down and take down child abuse material. According to Portnoff, the same concept can be applied to abused AI models.
“It’s not happening right now,” he said. “But I think it’s something that can and should be done.”