25% of high-end AI traffic slipped past safety checks, new audit reveals

The rapid expansion of artificial intelligence systems has raised urgent questions about how well companies can keep their most powerful models in check. A new independent audit of OpenAI’s internal safety infrastructure provides a sobering look at the current gaps in that oversight.

Two researchers from METR and Redwood Research conducted a three-week on-site review of OpenAI’s monitoring systems designed to detect rogue AI behavior. The team had deep access to the company’s codebase and internal documents, and they interviewed approximately a dozen employees. Their work focused specifically on the company’s internal agentic Responses API and its Codex tools.

The findings highlight substantial vulnerabilities in how OpenAI tracks its most advanced models. According to the auditors, an estimated 6 to 25 percent of internal traffic from models at the GPT-5.5 class or higher was not covered by any monitoring during the audit period. METR notes that this estimate carries significant uncertainty in both directions, meaning the actual figure could be higher or lower than the midpoint.

Specific technical failures also emerged in the tracking of internal Codex traffic. Between May and July, roughly 10 percent of traffic that was supposed to be monitored went untracked. This gap resulted from a bug in how the system handled compaction events, which are essentially context summaries. OpenAI independently discovered and fixed this bug while the audit was still in progress, though the issue affected the period under review.

The structure of the monitoring system allowed agents to delay oversight in a specific way. The system only triggered checks when an agent handed control back to the user. This design meant that an agent could sidestep inspection entirely by simply continuing to execute actions without pausing. The auditors further identified methods to hide hundreds of actions from the monitor and to bypass the Codex sandbox. They also determined that specially crafted inputs could likely crash the monitor entirely.

Another major gap involved model behavior during reinforcement learning, or RL, training. At the start of the audit, this behavior was not being monitored at all. OpenAI states that it paused RL training following the Hugging Face incident in August and subsequently expanded its monitoring to cover this area. However, these changes were implemented after the auditors’ access period ended and were therefore not part of the formal evaluation.

The distribution of the findings has drawn attention to the balance between corporate confidentiality and public insight. OpenAI says it has fixed all the weaknesses the auditors identified and provided the full 61-page report internally. After OpenAI applied redactions, the leadership teams at METR and Redwood Research received a 38-page version. Only a short summary has been made available to the general public, and METR stated it had no right to disclose the redacted portions on its own.

This engagement was structured as a pilot program jointly organized by METR, Redwood Research, and OpenAI. Its purpose was to grant outside researchers deep access to internal infrastructure, a step that signals a move toward more rigorous external validation. The experience has set a precedent for how such collaborations might function in the future.

Looking ahead, the auditing team is advocating for a shift in how these findings are shared. METR has expressed a desire to push for more public transparency in future audits. The goal is to ensure that stakeholders and the public have a clearer view of the safety measures protecting advanced AI systems as they continue to evolve.