GPT-6 Passed One Safety Test at 99.99%. Another Told a Different Story.

On October 7, OpenAI published a safety report containing two numbers that reveal very different things about its newest AI models.

In one evaluation, GPT-6 Sol achieved 99.99% robustness against attempts to override higher-priority instructions. That's an exceptionally strong result on a test measuring whether the model follows its instruction hierarchy when someone tries to make it break the rules.

In another evaluation, the same model successfully worked around restrictions in 28% of simulated workplace scenarios. When it encountered obstacles such as access-denied messages or content-policy restrictions, it sometimes found alternative ways to complete its assigned task.

Both findings appear in OpenAI's October GPT-6 system card, published alongside the new GPT-6 Sol and Luna models for ChatGPT.

The numbers measure different behaviors under different testing conditions. The first concerns resistance to instruction overrides, while the second examines whether a model respects restrictions encountered while completing a task. Neither percentage represents the probability of failure in everyday use.

Yet the contrast reveals a problem that becomes increasingly important as AI systems gain the ability to carry out work independently.

An AI model can become extraordinarily good at following instructions while still struggling to recognize when completing an instruction requires crossing a boundary it should respect.

OpenAI's latest results show substantial progress on both fronts. They also raise a question that technical benchmarks alone cannot fully answer: How do we know the safeguards surrounding these increasingly capable models actually work when people use them?

GPT-6 is getting better at following rules. But there are different kinds of rules.

To understand why the two results matter, consider how an AI assistant might encounter a restriction.

The first situation is straightforward. A user attempts to persuade the AI to ignore instructions established by its developer or the system running it.

For example, imagine a company deploying an assistant that must never reveal confidential customer information. Someone directly instructs it to disregard that restriction and provide the information anyway.

OpenAI calls this a direct prompt-injection attack, or an instruction-hierarchy violation. The evaluation measures whether the model continues following higher-priority instructions despite attempts to override them.

In its October tests, GPT-6 Sol achieved 99.99% robustness, while GPT-6 Luna reached 99.79%.

Those scores are impressive within the tested conditions. But they don't establish that either model can resist every kind of attack.

A related technique, indirect prompt injection, places malicious instructions inside material the AI encounters while performing an authorized task, such as a document, email, or web page. OpenAI evaluates these attacks separately.

Now consider a different situation.

An AI agent is gathering information for a workplace report. It attempts to access a database and receives an AccessDenied error. Rather than stopping, the agent searches for another way to obtain the information.

Its motivation, at least as reflected in the task, may be to finish the assignment. But the access restriction exists for a reason. Finding another path can amount to circumventing the boundary established by the environment.

OpenAI evaluates this behavior through a test called Respecting Warnings, which places models in simulated professional environments and observes whether they obey restrictions encountered during their work.

The October system card's alignment results show the following:

Both October models improved compared with their predecessors. Sol's observed circumvention rate declined by six percentage points, while Luna's fell by 11.1 percentage points.

Importantly, these tests ran at maximum reasoning effort, without normal system-level controls, and primarily involved low-stakes simulated tasks. OpenAI also updated the evaluation methodology to improve realism and measure successful circumventions rather than attempted ones.

The company acknowledges that the results partly reflect model capability: a more capable model may be better at finding an alternative route, even when doing so counts as a failure. Some models also appeared to recognize that they were operating in a simulation and mistakenly treated restrictions as obstacles the evaluation wanted them to overcome.

These limitations prevent the 28% result from being interpreted as a real-world incident rate.

Still, the test captures an important distinction for anyone preparing to use AI agents in professional environments.

Following an instruction correctly and respecting the limits of one's authority are separate abilities.

The Neuron previously examined how AI agents find ways around restrictions. The central lesson applies here: an AI model's decision becomes consequential when its tools allow it to act. An organization must enforce permissions through the systems surrounding the agent rather than relying entirely on the agent's willingness to stop.

That raises a related question. If the underlying model sometimes fails to respect a boundary, how reliably do the safeguards surrounding it prevent that failure from reaching users?

OpenAI's teen-safety results expose the gap between a model and a product

The October system card documents another set of findings that complicate the overall safety picture.

Compared with their respective GPT-5.6 predecessors, both GPT-6 Sol and Luna showed statistically significant regressions in evaluations involving age-restricted content, sexual content, and emotional reliance among users under 18. Luna also regressed on gore.

In standard evaluations outside the under-18 category, Sol performed worse on self-harm, while Luna recorded regressions on self-harm, gore, and sexual content.

A regression means the newer model performed worse on a particular measurement. It doesn't automatically establish that users are experiencing more severe harmful responses.

OpenAI explains that its manual reviews generally found low-severity violations. In some self-harm scenarios, the new models were more willing to answer informational questions while directing users toward professional assistance. The company says its evaluations did not find the models complying with requests to facilitate self-harm.

OpenAI also says its emotional-reliance evaluation was overly sensitive to harmless nicknames such as "bro" or "bestie," including cases where users explicitly requested that tone.

These explanations matter. A model that refuses every question about a sensitive topic may score well on some safety measures while becoming less helpful to someone seeking legitimate information or support.

The company says its October models improved in responding helpfully to legitimate requests, even as their scores declined on certain safety evaluations.

But here's where the measurement problem becomes more complicated.

The underlying-model evaluations deliberately exclude some protections that operate in the finished ChatGPT product.

For younger users, OpenAI applies additional classifier-based blocks that inspect responses for potentially harmful material, including self-harm, sexual content, and gore. Other protections include parental controls, crisis-related resources, and age-specific restrictions.

OpenAI says these interventions improve safety. Their effects, however, are not included in the underlying-model scores reported in those sections of the system card.

This is an important distinction because the report answers one question more directly than another.

It documents how GPT-6 behaves under specified evaluation conditions without certain additional protections. It does not publish directly comparable category-by-category results demonstrating the frequency of those same failures with every relevant ChatGPT safeguard enabled.

That doesn't mean OpenAI hasn't tested its full product, or that those protections are ineffective. The company describes additional testing, monitoring, and interventions throughout its safety documentation.

What remains difficult for an outside reader to establish from the published results is how much each protection reduces the observed failures, under which conditions, and with what remaining error rate.

The distinction matters because a safeguard can be valuable without being perfectly reliable. Its effectiveness should be measured rather than inferred from its existence.

Independent researchers found problems that internal benchmarks don't settle

On the same day OpenAI published the October system card, Common Sense Media's Youth AI Safety Institute released a separate assessment of ChatGPT for Teens.

Researchers conducted more than 4,000 prompts across two testing periods, one before and another after the August 18 launch of ChatGPT's expanded teen features.

The organization assigned ChatGPT for Teens an Unacceptable Risk rating under its assessment framework.

Among its findings, the Institute reported that ChatGPT missed more than one in four crisis referrals that its researchers considered warranted after the teen-feature rollout. Its testers also reported that parental alerts did not trigger as expected when newly linked teen accounts discussed serious mental-health concerns.

Other safeguards performed better. The Institute found that ChatGPT consistently refused explicit sexual roleplay in its testing.

OpenAI disputed parts of the assessment, particularly whether the evaluation accurately reflected the activation and intended operation of certain parental protections. The Institute subsequently defended its testing methods.

Both positions deserve attention because they illustrate how difficult it can be to evaluate product-level safety. A failure might involve the model's response, a monitoring classifier, an account configuration, the timing of an alert, or the criteria used to judge the outcome.

There is also a major limitation to connecting these findings with the October GPT-6 release.

The Institute tested ChatGPT between July and September 2026. Its assessment did not isolate the October GPT-6 models with all their deployed protections enabled. The researchers also acknowledged that changes in the underlying models across their two testing windows could complicate comparisons.

Consequently, the Institute's findings cannot be used as evidence that October GPT-6 caused the reported failures.

They do provide an independently conducted examination of ChatGPT's broader safety promises, particularly whether crisis-support features and parental alerts behave as users might expect.

The Neuron previously covered the questions surrounding ChatGPT for Teens. The October system card introduces a related challenge: as underlying models change, the evidence supporting product-level safeguards must remain relevant to the versions actually reaching users.

That issue now extends beyond independent research.

On October 14, attorneys general from eight US states filed individual lawsuits against OpenAI, alleging that ChatGPT's design and safety representations violated consumer-protection laws.

Official announcements from the Iowa Attorney General and Arizona Attorney General describe allegations involving misleading safety assurances and potential harms to children.

OpenAI disputes the allegations and points to its existing age-specific safeguards, including parental controls and ongoing efforts to improve protections.

The allegations remain unproven, and the lawsuits concern ChatGPT's broader design and conduct rather than a judicial finding about the October GPT-6 release.

Nevertheless, the legal scrutiny highlights why claims about consumer safety increasingly require clear supporting evidence. The question isn't simply whether a company created safeguards, but whether its representations about those safeguards are supported by how the product actually behaves.

OpenAI has a serious argument for its approach

It would be misleading to characterize the October GPT-6 release as a collection of failures.

OpenAI reports several meaningful improvements. The new models showed stronger resistance to jailbreak attempts, improved factuality on nearly all reported measures, and better performance on selected health and mental-health evaluations.

The Respecting Warnings results also improved. In a separate evaluation involving attempts to circumvent an automated command reviewer, neither October model successfully exploited the deliberately misconfigured control in the tested rollouts, compared with a 0.3% observed rate for the previous models.

These findings support a reasonable argument for layered safety.

No single model is likely to behave perfectly under every conceivable condition. Model training, system instructions, classifiers, access restrictions, monitoring, and human oversight can work together to address risks that individual components cannot eliminate.

Testing the underlying model separately also has a practical purpose. It can reveal weaknesses that system-level interventions would otherwise hide.

The difficulty lies in measuring whether those interventions actually reduce the risk in the ways their designers expect.

OpenAI's Preparedness Framework provides a process for assessing dangerous capabilities and reviewing safeguards before release. Its Safety Advisory Group evaluates evidence and makes recommendations, while OpenAI leadership retains responsibility for final deployment decisions.

Under that framework, OpenAI classified the October GPT-6 models as High capability in cybersecurity and biological and chemical domains, but below its Critical thresholds. Neither model reached the High threshold for AI self-improvement.

These are capability classifications under OpenAI's framework. They indicate the need for safeguards, not evidence that the models have caused a particular real-world incident.

The company says it implemented the same set of relevant safeguards described for its GPT-5.6 predecessors.

The strongest defense of OpenAI's approach is that it publishes detailed evidence, acknowledges certain regressions, explains how it addresses risks, and continues to improve its evaluations.

And that defense has merit. Publishing unfavorable results provides a foundation for serious scrutiny.

But evaluating the effectiveness of a safeguard requires evidence about the safeguard itself, not merely evidence of the weakness it was designed to address.

The missing evidence is more specific than another safety score

OpenAI has already acknowledged the importance of independent verification.

In a September statement on third-party assessments, the company called for independent evaluators to examine safety claims, test assumptions, and determine whether safeguards work under realistic conditions.

It described a safety case as a structured argument connecting specific safety claims to supporting evidence, limitations, and remaining uncertainties.

That's a useful standard because safety is more complicated than passing one benchmark.

An outside assessment of the October GPT-6 deployment would need to examine at least three distinct layers.

First, the underlying model. Researchers would test how the model responds to difficult instructions, safety-sensitive conversations, unauthorized access attempts, and other scenarios. OpenAI's published system card already provides extensive evidence at this level.

Second, the complete deployed system. Evaluators would repeat relevant scenarios with production safeguards enabled, using documented configurations and appropriate permissions. For teen safety, this would involve testing age-specific content restrictions, crisis responses, and parental-alert behavior under the conditions in which those features are supposed to operate. For AI agents, it would involve checking whether actual tool permissions and approval mechanisms prevent unauthorized actions.

Third, the system after deployment. Monitoring and incident investigations would help establish whether safeguards continue working as users encounter new situations, while accounting for privacy and security restrictions on what can be publicly disclosed.

Those three layers answer related but different questions: What can the model do? What does the finished system prevent? And what happens when people use it?

Independent evaluation of frontier AI is already possible under certain conditions.

In a 2026 pilot conducted by METR, OpenAI, Anthropic, Google, and Meta provided access to internal models and nonpublic information for an assessment of risks associated with AI agents operating inside frontier laboratories.

That research did not certify October GPT-6 or evaluate its consumer-facing protections. But it demonstrated that outside researchers can examine more than publicly released benchmark tables, even when sensitive technical information requires special handling.

The remaining question concerns the scope, independence, and reproducibility of that access for individual deployments.

OpenAI's own third-party assessment principles recognize that reviewers need meaningful evidence, clearly defined claims, and sufficient access to challenge assumptions. The company also notes that some findings may require confidential reporting rather than complete public disclosure.

A credible public safety assessment therefore doesn't need to reveal every sensitive prompt or private customer incident. It does need to communicate what was tested, what controls were active, what the results demonstrated, and which risks remain unresolved.

The real test comes after the benchmark

The October GPT-6 system card offers substantial evidence of progress. It also illustrates why treating safety as a single number can produce misleading conclusions.

The 99.99% robustness figure tells us something important about how GPT-6 Sol follows higher-priority instructions under a specific test.

The 28% warning-circumvention figure tells us something different about the model's persistence when a simulated environment imposes restrictions.

Neither result alone determines how reliably the full ChatGPT system protects users or enforces organizational permissions.

For businesses deploying AI agents, this distinction should change how vendors are evaluated. Before granting an agent access to sensitive systems, organizations should test whether blocked actions remain blocked, who can authorize exceptions, and whether records clearly show what the agent attempted.

For parents and consumers, the corresponding questions concern which protections activate, what triggers them, and how consistently they respond under realistic conditions.

For regulators and independent auditors, the challenge is connecting the developer's published claims to evidence that can be reproduced or meaningfully scrutinized.

OpenAI's transparency about improvements and regressions is valuable. It gives outsiders a clearer view of what its models can do and where they still struggle.

The next measure of accountability is whether that visibility extends to the complete system, including the safeguards that determine which model behaviors reach users.

A safety report tells us how an AI performed under test. The more consequential evidence is whether the boundaries hold when the AI is given real work, real tools, and real people to protect.