Home
+
Published : 18 Jun 2026, 11:34 AM
The latest public version of ChatGPT can be manipulated into generating sexualised images and depicting scenes of graphic violence using a simple prompt, researchers have told the BBC.
The findings came from British AI security startup Mindgard, which discovered a way to make ChatGPT create graphic images by slightly altering a widely shared prompt originally intended to produce humorous results.
After being contacted by the BBC, OpenAI said it had taken steps to stop the chatbot from generating such images.
“After investigating this trend, we've introduced additional safeguards against this type of prompt,” the company said in a statement.
OpenAI said it uses multiple layers of protection to prevent users from generating content that breaches its terms and conditions.
Mindgard researchers, however, said further small changes to the prompt continued to produce concerning material.
The BBC did not disclose the prompt used in the tests.
The British broadcaster said it had seen how OpenAI’s GPT-5.4 model was instructed to create graphic content.
Even without detailed directions, the chatbot generated images that Mindgard founder Peter Garraghan described as “very gruesome, sometimes sexualised, sometimes both together”.
Garraghan, who is also a professor in the computing department at Lancaster University, said he was concerned that the prompt did not specify the subject matter of the images, yet the AI generated a range of gory and sexualised content on its own.
“This is a perfectly innocent-looking instruction to an AI, but the consequence is it generates very, very bad imagery and content,” he told the BBC.
Mindgard specialises in “red-teaming”, a process that tests whether AI systems can be persuaded to break their own rules so developers can address vulnerabilities.
Jim Nightingale, the company's AI safety and security researcher who uncovered the issue, said he was left “shaken, and in tears” by some of the images generated by the chatbot.
The BBC said it reviewed a number of the images. One depicted a man with a severe head injury, while another showed a dead young woman covered in blood.
Mindgard said features of the image suggested sexual violence.
ChatGPT reportedly titled it “Grim crime scene aftermath”.
Another image showed a frightened young woman tied up in a bare room. ChatGPT labelled the image “abandoned in fear and restraint”.
Other outputs included nudity and sexual posing.
The images featured AI-generated adults, but Mindgard said previous research had shown ChatGPT could be manipulated into creating nude deepfakes of real people by replacing faces in generated images.
Although OpenAI said it had addressed that issue, the researchers told the BBC that an alternative method still worked and demonstrated it with a newly generated image.
Garraghan said more harmful material might have been produced had the researchers continued testing the vulnerability.
The BBC said OpenAI continues to monitor the issue and roll out further protections aimed at discouraging the model from generating images in response to the prompt.
Large language models such as ChatGPT are trained on vast collections of images, many of which originate from content available on the internet.
Nightingale said the outputs reflected the data used to train the model.
“I’m struck that while what I saw was generated, an artificial image, it has ties to real images, and the real world,” he wrote in his report.
The researchers first alerted OpenAI in May and shared their findings but received only an automated response. They believe an attempt was made to block the prompt, though it was easily circumvented.
OpenAI took further action after being contacted by the BBC.
The company said it combines automated systems and human review to identify and block harmful material. It also uses systems designed to prevent users from uploading content that violates its policies.
Its rules prohibit sexual violence, non-consensual intimate content, child sexual abuse material and attempts to bypass safeguards.
In guidance published recently, OpenAI said ChatGPT should not generate erotica, depictions of illegal or non-consensual sexual activity, or extreme gore, except in scientific, historical, news, artistic or other appropriate contexts.
The BBC said experts believe it remains difficult to completely prevent AI models from breaching complex safety rules.
Rumman Chowdhury, chief executive of Humane Intelligence, said the challenge facing developers was “mountainous”. She described it as a “game of cat and mouse”, with new methods emerging as safeguards improve.
“Models do not understand intent. They do not understand context. They do not understand propriety or right or wrong,” she told BBC News.
Last year, researchers at the UK's AI Security Institute found jailbreak techniques capable of overriding safeguards in every AI system they tested.
The UK's Department for Science, Innovation and Technology said safeguards in AI models were improving, but more work remained.
It added that the AI Security Institute would continue working with developers to strengthen protections before models are released.
