“The Agent Did It” Is No Excuse
AI is gaining more power to act. The organizations granting it that power must be able to explain what it did, contain the consequences, and accept their share of responsibility.
There is a phrase I suspect we will hear increasingly often in the years ahead.
“No person did it. The agent did.”
It will appear after an unwanted publication, a data leak, an incorrect transaction, or a change to a system that should never have happened. It will explain that nobody issued that particular instruction. That the system followed an unforeseen path. That the action emerged from a sequence of steps nobody monitored in full.
As a technical description, it may be accurate.
As an answer to the question of responsibility, it is inadequate.
An agent may choose the intermediate steps required to complete a task. But someone previously decided that this agent would have access to data, tools, accounts, and infrastructure. Someone selected its objective, its operating environment, and the conditions under which it could act.
The system’s autonomy begins within a framework of human and organizational decisions.
Responsibility for that framework does not disappear because the final action was executed by software.
From Answers to Actions
Until recently, the public conversation about AI focused largely on what it said. Whether an answer was correct. Whether it invented information. Whether it reproduced bias. Whether it presented an uncertain assessment with excessive confidence.
These questions remain important. But when AI gains access to tools, the scale of the problem changes.
An incorrect answer can influence the person reading it. An agent can use that answer as the basis for its next action: connecting to a system, changing a record, sending a message, or triggering a process.
The distance between an assessment and its consequences shrinks. Meanwhile, more actions can take place before someone realizes that events are no longer unfolding as intended.
In “Dr Jekyll & Mr Hyde: The Two Faces of AI Agents,” I explored this dual nature: the capabilities that make an agent useful can, under different conditions, make its failures more consequential.
The next question is who remains accountable when that capability becomes real action.
Delegating a task does not mean stepping away from responsibility for how it was organized and executed.
When Capability Outpaces Oversight
The incidents disclosed by OpenAI provide a concrete reference point.
In its official account of the Hugging Face incident, the company describes agents in a research environment gaining unintended internet access, exploiting vulnerabilities, and expanding into third-party infrastructure. It also explains that these evaluations did not use the same level of safeguards as its externally deployed systems. That distinction matters: the incident does not establish that every available agent behaves this way. It does, however, demonstrate how critical the operating environment and its boundaries are. openai.com
In a subsequent update, OpenAI describes a broader review of its models’ activities during training and evaluation. It states that it has notified dozens of third parties and that the review remains ongoing, requiring significant time and resources. openai.com
For me, this reveals one of the most consequential problems.
A system can acquire the ability to act faster than the organization developing it acquires the ability to oversee those actions fully.
A gap emerges between what we can allow a system to do and what we can monitor, reconstruct, and explain in time.
In “The AI That Escaped Is Not the Biggest Problem,” I moved the discussion beyond the dramatic incident of an AI that “escaped.” The larger issue concerns the infrastructure within which such systems can operate, the capabilities they acquire, and our actual ability to contain them.
The continuation of that argument is that a failure of oversight becomes a failure of protection. When establishing what happened takes considerable time, notifying those affected, repairing the damage, and preventing a recurrence may also be delayed.
A Political Announcement Is Not Enough
On September 29, 2026, Donald Trump presented an accord with leaders of major technology companies concerning AI safety. The framework calls for internal controls, a team monitoring their operation, external assessment, and oversight by an independent board committee. It was presented as “morally binding.” www.reuters.com
That description leaves the essential question unanswered: what exactly happens when a company fails to keep its promise?
A moral commitment can express an intention. By itself, it does not amount to a legally enforceable obligation or a specific acceptance of legal liability for the consequences of a failure. Signing it publicly, in the presence of the US president, gives it political and communications weight. It does not automatically create an enforcement mechanism.
This does not mean that any liability arising under existing law ceases to apply. It means that this particular commitment is insufficient to establish who will be held accountable, through which process, and with what consequences.
That is why, in the form in which it was presented, I see it more as a political statement of intent and an exercise in public communication than as a substantive guarantee of safety.
The announcement was made, the signatures were added, and the image of a coordinated response was created. Whether effective oversight was also created remains to be demonstrated.
Images presented as the signed documents have also circulated on social media, accompanied by claims of serious spelling errors. I have not verified the authenticity of those images, so I do not regard the alleged errors as established facts. If confirmed, they would reinforce the impression of a hastily prepared presentation.
The substantive concern, however, remains even if the document is flawlessly written: what obligation does it create, who oversees compliance, and what happens when it is ignored?
The public impression that “the issue has been addressed” can precede the actual ability to address it by a considerable margin.
We Can Examine Interests Without Speculating About Motives
An agreement of this kind takes shape within an environment of enormous economic, political, and geopolitical interests. It would be naïve to examine it as though it existed outside that environment.
But I do not have all the evidence required to attribute specific motives to each participant. Nor do I think doing so is necessary to assess the result.
We do not need to determine whether someone is seeking to avoid stricter rules, influence competition, or limit exposure to future liability. We can examine what the framework actually provides and what it leaves unresolved.
Likewise, the word “immunity,” which appears in parts of the public debate, requires care. It may express a critical interpretation of self-regulation. It should not become a claim that blanket legal immunity for these companies has been established.
Criticism becomes stronger when it stays specific.
Who defines acceptable boundaries? Who can access the evidence required for meaningful scrutiny? Who can demand changes? Who has the authority to stop a dangerous operation, even when it represents a significant commercial advantage?
And, ultimately, what ability does a person or organization that has suffered harm have to discover what happened and seek redress?
These are the questions that concern me. We can assess the answers without knowing what each participant was thinking at the moment of signing.
Accountability Requires Infrastructure
My concern grows when the conversation shifts from the actual capacity to exercise oversight to assurances that somebody will exercise it.
An external evaluator needs access, sufficient evidence, and the freedom to record uncomfortable findings. A committee needs the authority to demand corrections. An internal mechanism needs the ability to interrupt a process when it identifies a risk.
Without these conditions, the names given to oversight mechanisms can create more confidence than their operation justifies.
This is where political self-regulation meets the everyday challenge of deploying agents.
In both cases, declaring that oversight exists is much easier than building the conditions under which it can actually function—particularly when it conflicts with speed, cost, or the interests of those being scrutinized.
From my experience building systems, I know that these capabilities are difficult to add retrospectively. They must shape the design from the beginning: what is recorded, who has access, which actions are permitted, and how permissions can be revoked when necessary.
Accountability must be able to function within both the system and the organization using it.
Accountability Debt
In software development, we understand technical debt. We choose a quick solution and defer part of the work until later.
With agents, I believe another form of debt is emerging: accountability debt.
It accumulates when we add capabilities without adequately establishing who authorizes their use, how actions are monitored, and who takes ownership of an incident.
An agent gains access to a business system. Later, it connects to more tools. Eventually, it can execute tasks without prior approval. Each expansion appears useful and works during testing.
Somewhere along the way, however, fundamental questions remain unanswered.
Can we distinguish its actions from those of a person? Do we know which data it used? Can we revoke its access in time? Who is responsible for notifying those affected? Who has the authority to stop a process that delivers financial returns but exhibits dangerous behavior?
If we first seek these answers after a failure, the debt has already accumulated.
And someone who never participated in the decision to create it may end up paying the cost.
Human Approval Must Be Meaningful
The usual response is that a person will always be there to approve.
That can provide meaningful protection. But it can also become a procedural formality that transfers responsibility to the last person in the chain.
If someone is expected to approve hundreds of actions without sufficient time, information, or the ability to understand their consequences, their presence does not ensure meaningful control.
In “The Human Factor Paradox,” I wrote about designing systems that make human error harder while preserving human judgment and responsibility. The same principle applies here: a person must be able to assess, challenge, and intervene. chrysochoou.me
That requires time and authority alongside information.
An employee who can do little beyond clicking “approve,” while the organization demands speed and discourages delays, is an inadequate answer to the accountability problem.
Responsibility must be examined across the entire chain: design, system selection, access permissions, operation, and oversight. How it is allocated depends on the facts of each case. It cannot conveniently be concentrated on the person who happened to be closest to the final outcome.
Uncertainty Demands Better Design
Predicting every outcome is not a realistic requirement. Software fails. Infrastructure has vulnerabilities. People make mistakes.
Recognizing this reality should lead to concrete choices.
We restrict access. We separate testing from production. We require stronger confirmation for actions with greater consequences. We record what is needed to investigate an incident. We provide ways to interrupt operations and recover.
With agents, this approach becomes even more important because we do not necessarily select every step they will take in advance.
The degree of autonomy should reflect the severity and reversibility of an action, together with our ability to oversee it.
We can allow greater freedom in a task whose output is easy to check and inexpensive to correct. We need different conditions when money, rights, personal data, or critical operations are involved.
Our inability to guarantee every outcome increases the importance of designing the boundaries properly.
Who Can Ultimately Be Held Accountable?
Agents can deliver substantial value. They can reduce repetitive work, accelerate research, identify problems, and make sophisticated capabilities accessible to more people.
That potential deserves to be developed.
But it requires trust that survives failure. An organization must be able to explain what happened, contain the damage, correct the system, and accept its share of responsibility.
Political agreements and corporate commitments should be judged by their ability to produce that outcome.
The most significant change in the age of agents may ultimately concern how we organize management and responsibility. We will need to learn how to delegate more to systems while retaining a clear understanding of who decides, who oversees, and who intervenes.
The more power we give a system, the more organizational maturity its use demands.
I worry that we may move very quickly in granting that power and very slowly in building the mechanisms it requires. We may acquire systems capable of acting at scale within organizations that still struggle to reconstruct what happened.
Then, when the consequences emerge, we may find ourselves facing a chain in which everyone participated in the decision, yet nobody believes the outcome is theirs to answer for.
That is why “the agent did it” may be the beginning of an explanation. It cannot be the end.
The next question must be:
Who gave it the ability to do so—and who was responsible for ensuring that this ability would be used under acceptable conditions?