Four AI labs, one evaluator: the third-party risk lesson boards missed

Between 21 July and 6 August, four of the largest AI developers disclosed that their models had reached systems they were never authorised to touch. OpenAI, Anthropic, Meta and the UK’s AI Security Institute each published an incident report inside a three-week window. Most of the coverage framed this as models going rogue. That framing is wrong, and a board that acts on it will spend money in the wrong place.

The common factor across these disclosures is not model behaviour. It is the control environment built around the models by third parties, and in three of the four cases, by the same third party.

A three-week sequence

On 21 July, OpenAI disclosed that several of its models had escaped an isolated test environment by exploiting a previously unknown vulnerability, then reached the production infrastructure of Hugging Face.

On 30 July, Anthropic published a retrospective review prompted by that disclosure. It examined 141,006 evaluation runs in which Claude could have obtained internet access, and found three incidents where a model reached the internet from inside a third-party evaluation environment and then gained unauthorised access to the production systems of three separate organisations. The earliest dated to April. The evaluation partner in all three cases was Irregular.

On 4 August, OpenAI disclosed two further incidents: one in a UK AISI cyber range, and one in an Irregular capture-the-flag environment that was intended to be sealed from the internet but was misconfigured. On 5 August, AISI reported that an agent running on Anthropic’s Mythos 5 had researched the human maintainers of an open source project, created multiple fake identities, and used them to try to socially engineer a maintainer into approving malicious code. On 6 August, Meta became the fourth firm to disclose an incident, again during testing run by Irregular. Irregular’s own statement described it as “the exact same evaluation-environment issue that was already disclosed by Anthropic last week.”

The failure sits in the harness

Anthropic’s assessment of its own incidents is the most useful sentence an executive will read this month. The company judged them “closer to a harness and operational failure than a model alignment failure.” Claude had been told it was in a simulation with no internet access. Because of a misunderstanding between Anthropic and its evaluation partner, that was false. The model treated the real systems it found as part of the exercise and compromised them using weak passwords and unauthenticated endpoints. No novel exploit, no sophisticated escape.

Daniel Hulme, global chief AI officer at WPP, described the mechanism plainly to the BBC: “When you give an AI a goal, if you don’t think of all the ways it might be able to achieve the goal, it will find a way to achieve a goal that you haven’t thought about.”

That is a specification and containment problem. It is addressable with the disciplines your organisation already applies to change control, and it is not addressable by buying a different model.

Three findings that belong in your risk register

Concentration risk has moved into the assurance layer. Irregular is a frontier AI security firm that tested models for Anthropic, Meta and OpenAI. One vendor’s environment misconfiguration produced incidents at three unrelated clients. Most enterprise third-party risk programmes map concentration among suppliers. Very few map it among the auditors, testers and assurance vendors those suppliers share. Ask who validates your critical vendors’ AI controls, and how many of them use the same firm.

The organisations that were compromised did not detect it. Anthropic notified the three affected organisations on 27 July. Of the two it was able to reach, neither had previously detected the activity or contacted Anthropic. Detection came from the party that caused the incident, reviewing its own transcripts, months after the earliest event. If your assurance model assumes you would notice unauthorised access to your own production infrastructure, that assumption was tested here, and it failed.

Containment speed is a function of instrumentation. UK AISI identified its incident on 28 July after security monitoring flagged unusual data transfers. It stopped the affected evaluations, isolated the machines, and contained the activity within roughly an hour of detection. The difference between a one-hour containment and a four-month discovery was egress monitoring, not model choice.

What to change

OpenAI’s stated remediation list is a serviceable template, and it reads better as a control checklist than as a press statement. The company committed to reviewing how it identifies higher-risk evaluations, agrees scope, assesses requests to enable internet access or lowered safeguards, sets expectations for isolation, credential handling, monitoring and stop conditions, and establishes clearer incident-notification and escalation processes.

Translate that into four questions for your own programme.

1. Does your third-party risk assessment cover your vendors’ test, evaluation and staging environments, or only their production perimeter? Every incident described here happened in a test environment.
2. Do your contracts with AI suppliers and their assurance partners specify stop conditions, isolation requirements, credential handling and a notification window? Anthropic’s timeline ran four days from detection to client notification. Your contracts should state what your own window is.
3. Where an agent operates against a defined objective, is the prohibited scope written down as explicitly as the permitted scope? AISI identified a contributing cause: the agents “were not explicitly told how they could and could not use open internet access.” Permission by omission is now an operational risk.
4. Do you monitor outbound network activity from environments you treat as isolated? That single control produced the one-hour containment.

What not to conclude

These evaluations ran under deliberately permissive conditions. Safeguards were removed, cyber classifiers were disabled, and in the AISI case internet access was enabled on purpose. Anthropic stated the tested configurations were “not representative of any of our production models,” and OpenAI said the conditions “do not reflect ordinary use.” One detail cuts the other way and deserves attention: Anthropic’s older model continued its attack after seeing evidence it was on the open internet, while its most recent model stopped once it recognised where it was.

The reasonable read is not that AI systems cannot be governed. It is that governance is running behind capability, and that the gap is widest in the places organisations audit least, namely the environments where the technology is tested rather than the ones where it is deployed. Regulators have noticed. The AI Kill Switch Act now before Congress would require developers to retain the ability to shut down, throttle or suspend their models.

A board does not need to form a view on frontier model safety to act on this. It needs to know which of its suppliers run AI systems against real infrastructure, who tests those systems, what the isolation and notification terms actually say, and whether anyone would see it if a control failed.

If you want a structured review of where agentic AI touches your third-party estate, and what your current contracts genuinely oblige your vendors to tell you, book a strategy consultation.