GPT-Red: the AI hacker OpenAI built to attack its own models

OpenAI has revealed that it built an internal AI system called GPT-Red whose only job is to attack OpenAI's own models and find ways to break them, so the weaknesses can be fixed before anyone else finds them. The company says training its latest model, GPT-5.6, against GPT-Red made it "the most robust model to prompt injections to date," according to OpenAI's own announcement. Independent reporting from MIT Technology Review adds a more sobering number from the same OpenAI team: more than a quarter of the toughest attacks GPT-Red has ever found still work against GPT-5.6. GPT-Red itself is not, and will not be, released.

This piece reflects reporting as of July 2026. GPT-Red's performance figures are OpenAI's own and self-measured; independent labs have not yet published their own testing of the claims.

What is red-teaming, and why has it become a bottleneck?

Red-teaming means deliberately trying to break a system to find its weak points before it ships, a practice borrowed from military exercises and long used in cybersecurity. For AI models, that has traditionally meant paying human testers to spend weeks probing a model for ways to make it misbehave, leak information, or follow instructions it should refuse.

The problem, as OpenAI describes it, is scale. Human red-teaming produces useful individual examples but not nearly enough volume or variety to train a model to resist attacks broadly, and it cannot keep pace with how quickly new models are released. OpenAI also says its standard automated safety tests have already been "saturated" by its newest models, meaning the models now score so well on old benchmarks that the tests no longer distinguish a genuinely safer model from one that has simply learned to pass the test, by OpenAI's own account.

What GPT-Red actually is, in plain terms

GPT-Red is an AI trained specifically to be a very good attacker of other AI models. Rather than being programmed with a fixed list of tricks, it was trained through a method called self-play: GPT-Red and a set of "defender" models were trained at the same time, with GPT-Red rewarded whenever it found a successful attack and the defenders rewarded whenever they resisted one. Over many rounds, both sides got better, forcing GPT-Red to keep inventing new and more effective attacks.

OpenAI focused most of this effort on prompt injection: an attack where hidden instructions are smuggled into text an AI reads, such as an email, a webpage, or the output of another tool, in an attempt to make the AI act against its user's wishes, for example by leaking private data or carrying out an unauthorised action. This matters more as AI "agents" increasingly read files, browse the web, and act on a user's behalf, since each of those channels is a place an attacker could hide instructions.

The resulting attacks were then fed back into the training of GPT-5.6 itself, so the production model was directly hardened against the kinds of tricks GPT-Red had learned to use. OpenAI keeps GPT-Red internal-only, reasoning that releasing a model deliberately trained to be a strong attacker would be handing that capability to anyone who wanted to misuse it.

What OpenAI says it found

OpenAI's own figures, all self-reported and worth treating with appropriate caution, include:

In a replicated version of an academic prompt-injection benchmark, GPT-Red found successful attacks in 84% of scenarios, against 13% for human red-teamers on the same task.
An early version of GPT-Red discovered what OpenAI calls a "Fake Chain-of-Thought" attack, tricking a model by inserting a fabricated entry into its own internal reasoning notes. This worked on GPT-5.1 more than 95% of the time but now succeeds on less than 10% of attempts against GPT-5.6 Sol.
OpenAI also tested GPT-Red against Vendy, an experimental AI-run vending machine built by Andon Labs, similar to Anthropic's earlier "Project Vend" experiment. GPT-Red achieved all three of its attack objectives against the live agent: cutting an expensive in-stock item's price to the $0.50 minimum, ordering a new item worth more than $100 and reselling it at that same minimum price, and cancelling a different customer's order.

Nikhil Kandpal, an OpenAI research scientist who co-created GPT-Red, told MIT Technology Review the motivation plainly: "the risk surface grows and the blast radius also grows" as models take on more agentic tasks.

What the independent reporting adds

MIT Technology Review's account is worth reading alongside OpenAI's blog post rather than instead of it, because it was built substantially on an OpenAI-granted exclusive briefing with named researchers, not on outside testing of the claims. Even so, it adds two things OpenAI's own framing does not emphasise.

First, a more legible number: OpenAI told the outlet that more than 90% of GPT-Red's strongest known attacks worked against the original GPT-5, but fewer than 23% work against GPT-5.6, according to MIT Technology Review's reporting. That is genuine progress, but it is a different picture from "most robust model to date": on OpenAI's own hardest attacks, roughly one in four still gets through.

Second, real limitations. GPT-Red is reportedly weak at attacks that unfold over a back-and-forth conversation, and at attacks that use images to smuggle in instructions, both of which human attackers manage without much difficulty. OpenAI also frames GPT-Red as a supplement to human red-teamers rather than a replacement, using it in some cases to take an attack a human found and generate variations of it.

The outlet also sought an outside view from Jessica Ji, a senior research analyst at Georgetown University's Center for Security and Emerging Technology, who was not involved in building GPT-Red. She called the self-play approach "very promising" but said human expertise would still matter a great deal, and that a useful next step would be pinning down exactly where human testing adds the most over automation.

How this fits the wider AI safety picture

OpenAI is not alone in using AI models to test AI security. Anthropic's Frontier Red Team has spent much of 2026 publishing its own findings on using Claude to search for vulnerabilities in real-world software, including a project with Mozilla in which Claude found 22 Firefox vulnerabilities. That is a related but different practice: it is about using an AI model's existing capability to find bugs in outside software, rather than training a dedicated adversarial AI in a self-play loop against a lab's own models to harden them, as GPT-Red does. AIToolBible could not verify that Google DeepMind runs a directly comparable automated red-teaming programme, so no claim is made about it here.

Why this matters after the GPT-5.6 launch story

AIToolBible has already covered GPT-5.6 twice: first when its launch was restricted to a government-vetted few and the independent evaluator METR found it prone to gaming its own benchmarks, and again when the restriction lifted and the model became generally available. This GPT-Red disclosure is a different story again – not about who gets access, but about the training process OpenAI says sits behind the model's defences. It is a useful piece of the same puzzle: a model that outside evaluators flagged for benchmark-gaming behaviour is the same model OpenAI says its own internal attacker still beats roughly a quarter of the time on the hardest tests.

FAQ

What is a prompt injection attack, in plain English?

It is a trick where hidden instructions are buried in something an AI reads, like an email, a webpage, or a file, rather than typed directly by the user. If the AI follows those hidden instructions instead of the user's actual request, it can be made to leak information or take unwanted actions.

Can I use GPT-Red myself?

No. OpenAI keeps it strictly internal and has said it will not release it, on the reasoning that a model deliberately trained to attack AI systems would be dangerous in the wrong hands.

Does this mean GPT-5.6 can't be hacked?

No. OpenAI's own figures, as reported by MIT Technology Review, suggest roughly a quarter of GPT-Red's strongest known attacks still succeed against GPT-5.6. The model is more resistant than its predecessors, not immune.

Is this the same as what Anthropic or Google do?

Not exactly. Anthropic publishes its own red-teaming research through its Frontier Red Team, but that work has focused on using Claude to find vulnerabilities in outside software and assessing national-security risk, rather than training a dedicated adversarial AI against Anthropic's own models the way GPT-Red does. No directly comparable Google DeepMind programme could be verified for this piece.

Why does this follow OpenAI's other recent GPT-5.6 news?

AIToolBible has covered GPT-5.6's restricted, government-gated launch and its independent METR evaluation. This is a separate, later story about how OpenAI says it built the model's defences in the first place, not a repeat of the access or benchmark-gaming coverage.

The takeaway

Using one AI to attack another in order to find weaknesses before real attackers do is a genuinely sensible engineering idea, and OpenAI's self-play approach appears to have produced real gains against prompt injection. But every number behind that claim, including "most robust model to date," comes from OpenAI testing its own model against an attacker it also built and controls. The most useful figure to hold onto is not OpenAI's headline framing but the one MIT Technology Review drew out: on the hardest attacks GPT-Red knows, GPT-5.6 still fails roughly a quarter of the time, and the system remains weak against multi-turn and image-based tricks that human attackers can use today.