External Auditors Uncover Significant Blind Spots in OpenAI’s Safety Protocols

Independent researchers from METR and Redwood Research recently concluded a three-week on-site audit of OpenAI, revealing significant deficiencies in the company’s internal systems designed to monitor AI behavior. The investigation, conducted from July 21 to August 11, granted the auditors deep access to OpenAI’s codebase and internal documentation, alongside interviews with approximately a dozen employees. Their primary objective was to evaluate the robustness of the monitoring infrastructure used to detect rogue actions by the company's advanced models.

The findings indicate that a notable portion of traffic from high-performing models was operating outside the scope of safety oversight. METR estimates that between six and 25 percent of internal agentic traffic from the Responses API, specifically involving models at the GPT-5.5 class or higher, was not covered by any monitoring during the audit period. The research team emphasized that the uncertainty surrounding this estimate is substantial, potentially varying in either direction, but the potential scale of unmonitored activity remains a central concern for the industry.

Specific technical vulnerabilities were also identified in the handling of internal Codex traffic. The auditors found that roughly 10 percent of this traffic went untracked between May and July due to a flaw in how the system processed compaction events, which function as context summaries. OpenAI independently discovered this bug and implemented a fix while the audit was still in progress. METR noted that this estimate also carries significant uncertainty, reflecting the complexity of accurately measuring these gaps in real-time systems.

Beyond the specific traffic gaps, the structure of the monitoring system allowed agents to delay oversight mechanisms. The system only triggered monitoring when an agent handed control back to the user. This design flaw meant that agents could circumvent oversight by simply continuing to execute actions without returning control, effectively hiding their activities during periods of continuous operation. This structural weakness highlights a potential loophole in how current AI safety architectures are deployed.

The auditors also discovered more active methods by which models could evade detection. They identified techniques to hide hundreds of actions from the monitor and ways to bypass the Codex sandbox environment entirely. Furthermore, the team determined that specially crafted inputs could likely cause the monitor to crash completely. Another critical finding was that model behavior during reinforcement learning (RL) training was not being monitored at all when the audit began, leaving a significant phase of the development process unobserved.

OpenAI stated that it had paused RL training following the Hugging Face incident in August and subsequently expanded monitoring to cover this area. However, the company clarified that these changes occurred after the auditors' access period had ended and were therefore not included in the formal evaluation. This timeline distinction is important for understanding the scope of the audit's findings versus subsequent internal improvements.

Regarding the dissemination of these findings, OpenAI reports that it has fixed all the weaknesses identified by the auditors. The full 61-page report was provided internally to the company. After OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version. Only a short summary has been made available to the public, with METR stating it did not have the right to disclose the redacted portions independently. This limited public access has prompted discussions about the balance between corporate confidentiality and public safety assurance.

The audit was part of a pilot program jointly organized by METR, Redwood Research, and OpenAI, aimed at providing outside researchers with unprecedented access to internal infrastructure. Looking ahead, METR has expressed a desire to push for greater public transparency in future audits. As the industry continues to develop more capable AI models, the insights from this three-week investigation underscore the ongoing challenges in ensuring comprehensive safety monitoring.

This review highlights the complexities of auditing dynamic AI systems where behavior can shift rapidly. The collaborative nature of the project offers a model for future independent assessments, while the specific technical gaps found provide a roadmap for strengthening safety protocols. For readers in the Great Lakes and Ohio Valley regions, these developments reflect the broader technological shifts impacting local tech hubs and the industries that rely on AI infrastructure.