🔗 Share this article

Technology companies and child safety agencies will be granted authority to evaluate whether AI tools can produce child abuse material under new UK laws.

Significant Rise in AI-Generated Harmful Material

The declaration coincided with findings from a protection monitoring body showing that cases of AI-generated CSAM have more than doubled in the past year, growing from 199 in 2024 to 426 in 2025.

Updated Regulatory Framework

Under the changes, the government will allow designated AI companies and child safety organizations to examine AI systems – the foundational technology for conversational AI and image generators – and ensure they have sufficient protective measures to stop them from producing images of child exploitation.

"Fundamentally about preventing abuse before it occurs," stated the minister for AI and online safety, adding: "Specialists, under rigorous protocols, can now detect the danger in AI models promptly."

Tackling Legal Obstacles

The amendments have been implemented because it is illegal to create and possess CSAM, meaning that AI creators and others cannot generate such images as part of a testing regime. Until now, officials had to delay action until AI-generated CSAM was uploaded online before addressing it.

This legislation is aimed at averting that issue by enabling to halt the creation of those images at source.

Legal Framework

The amendments are being introduced by the authorities as revisions to the crime and policing bill, which is also implementing a ban on possessing, creating or sharing AI models designed to generate child sexual abuse material.

Real-World Impact

This recently, the official visited the London base of Childline and heard a simulated conversation to advisors involving a account of AI-based exploitation. The interaction portrayed a adolescent requesting help after facing extortion using a explicit deepfake of himself, created using AI.

"When I learn about young people experiencing extortion online, it is a source of extreme anger in me and rightful anger amongst parents," he said.

Concerning Statistics

A prominent internet monitoring organization stated that instances of AI-generated abuse material – such as webpages that may include numerous files – had significantly increased so far this year.

Instances of category A content – the gravest form of exploitation – increased from 2,621 visual files to 3,086.

• Girls were predominantly targeted, making up 94% of prohibited AI depictions in 2025
• Portrayals of infants to toddlers rose from five in 2024 to 92 in 2025

Industry Reaction

The law change could "constitute a crucial step to guarantee AI products are safe before they are released," stated the head of the internet monitoring foundation.

"AI tools have enabled so survivors can be targeted repeatedly with just a few clicks, providing offenders the capability to make possibly limitless amounts of sophisticated, photorealistic exploitative content," she continued. "Material which additionally commodifies victims' trauma, and renders children, especially female children, less safe on and off line."

Counseling Session Data

Childline also released details of support interactions where AI has been referenced. AI-related harms mentioned in the sessions comprise:

• Employing AI to evaluate body size, body and looks
• Chatbots discouraging children from consulting trusted adults about abuse
• Being bullied online with AI-generated material
• Online blackmail using AI-faked pictures

Between April and September this year, Childline delivered 367 support sessions where AI, chatbots and associated topics were discussed, four times as many as in the same period last year.

Half of the references of AI in the 2025 sessions were connected with psychological wellbeing and wellness, including utilizing chatbots for support and AI therapeutic apps.