News
OpenAI Launches Free Tool to Detect AI-Generated Child Sexual Abuse Material
OpenAI introduces a free CSAM detection tool to combat AI-generated abuse, offering support to child safety organizations and law enforcement.
OpenAI has unveiled a new tool aimed at detecting child sexual abuse material (CSAM), including content generated by artificial intelligence, as part of its expanded efforts to combat online child exploitation. The tool, available at no cost to vetted organizations through the OpenAI API, is designed to assist child safety organizations and law enforcement in identifying and addressing abusive material more effectively.
Advanced CSAM Detection Technology
The newly released image classifier is capable of identifying potential CSAM, including AI-generated material, by analyzing visual signs of abuse. It provides scores to help reviewers prioritize images that may require immediate attention. OpenAI collaborated with the National Center for Missing and Exploited Children (NCMEC) to train and evaluate the classifier for real-world deployment. This marks the first broadly available CSAM classifier with a specific signal for detecting AI-generated content.
OpenAI emphasized that this tool builds on its existing safeguards, which include prohibiting CSAM, training AI models to reject requests for abusive material, and taking action against accounts violating its policies. The company also introduced product warnings in ChatGPT to interrupt harmful behavior and implemented 24-hour suspensions for attempts to access abusive content.
Collaboration Across Organizations
Recognizing that child safety is an industry-wide issue, OpenAI is partnering with organizations such as the Tech Coalition and ROOST to share access to the new tool. The company is also participating in the Tech Coalition's Lantern program, which facilitates cross-platform collaboration by sharing signals related to online child exploitation. OpenAI expressed strong support for complementary tools developed by other organizations, such as Thorn’s Safer CSAM detection system.
Support for Law Enforcement and Developers
In addition to the image classifier, OpenAI is providing practical resources for developers and support for child-safety law enforcement investigators. These efforts align with the company’s Child Safety Blueprint, which outlines its commitment to detecting, blocking, and reporting abuse while empowering parents with technology that prioritizes child safety.
By offering these tools and fostering collaboration across industries, OpenAI aims to strengthen the global fight against online child exploitation and provide critical support to those working to protect children.