Capability scales. Accountability doesn’t.

We spend a lot of time measuring what these systems can do and much less time deciding who’s allowed to let them do it.

It’s 4:52 on a Friday, and a page is down.

Not the homepage. One operator’s page, on a platform of about a thousand sites running on a shared template. The operator has a prospect arriving Monday who was sent the link. They don’t know what a template is. They know their page is broken and that it worked on Wednesday.

I’ve been the person whose phone rings at that moment. The questions aren’t the ones you’d expect. Nobody is wondering whether the problem is technically solvable. On a platform like this, the fix is usually small, and someone can almost always work it out. The questions are harder: Who’s allowed to touch production at 4:52 on a Friday? Was the change that broke the page approved, and by whom? Will fixing this page affect the other thousand? Who tells the operator what happened, and what can they promise? If the fix causes another problem, who answers for it on Monday morning?

I built the web operations function at a national organization from scratch. That meant writing the documents that answer those questions: a release process, a support playbook, and an accountability document specifying who may make each kind of change and who must approve it. I also build with coding agents every day, on my own products and at work. Doing both has made one gap hard to ignore: we spend a lot of time measuring what these systems can do and much less time deciding who’s allowed to let them do it.

Two conversations

The public conversation about AGI is largely a conversation about intelligence: how capable systems are, how quickly that capability is growing, and which tasks they will take on next. When the conversation turns to work, it often treats a job as a sequence of thinking tasks, each with an input and an output, that a sufficiently capable model could perform one by one.

Inside an operations function, intelligence is rarely the bottleneck. I can’t remember an incident where nobody could work out what to do. Usually nobody knew whether they were allowed to act, or the person with authority couldn’t be reached, or the change hadn’t been approved and the approver had gone home, or two people each thought the other owned it. The fix was obvious. The authority to make it wasn’t.

The word that connects these problems is ownership. The owner of a change is the person who agrees in advance to answer for it when it goes wrong. That sounds simple until you ask what exactly they’re agreeing to own: the change itself, the risk it creates, or the decision to let someone else make changes like it. The documents I wrote were meant to make those relationships explicit. Leave ownership implicit, and 4:52 on a Friday is when you find out everyone was working from a different assumption.

Three documents

The release process defines who may change production and under what conditions. In the version I wrote, a change moves through named stages, each with a condition that must be met and a person responsible for confirming it. The change has to be tested outside production and reviewed by someone other than its author. There are windows when changes may go out and windows when they may not; late Friday afternoon is one of the latter, for the reason this essay opens with. A change to the template shared by every site needs different approval from a change to one site because its blast radius is larger. And every change records who made it, so when a page breaks on Friday, someone can trace what went out on Thursday and ask the right person.

Now put an agent in that process. It can make the change, run the tests, and review another agent’s work. In my experience that review can be better than a tired human’s. But the agent can’t be the person named in the process. Each stage is a gate where someone confirms that a condition has been met and accepts responsibility for that judgment. An agent can produce evidence that the condition is met; it can’t be the one who answers for accepting it. If an agent confirms its own gates, the process has no gates. I’ve seen people build exactly that and call it automation.

The support playbook defines who gets called and what they may say. It routes problems to the right people and separates promises support can make from commitments reserved for whoever owns the system. Support could tell an operator their page would be fixed. It couldn’t say when, or how, or promise anything beyond the fix, because those are commitments someone has to be able to keep, and the person on the phone didn’t have that authority.

An agent can be an excellent first responder. It’s patient, it has read the playbook, and it never forgets to log a ticket. It can also make a promise without understanding what that promise commits the organization to. The playbook exists to limit who may commit the organization to what, and an agent should be subject to the same limits. But it can’t be left to invent those limits in the moment. Someone has to decide in advance which commitments it may make, what evidence it needs, and when it has to stop and escalate.

The accountability document is the one I’d keep if I could keep only one. It’s a table. Each row is a kind of change the platform can undergo: content on one site or all sites, templates, integrations, domains, access. For each row, it lists who may propose, make, and approve the change, and who must be notified. Writing it was the most useful thing I did in that role, for an embarrassing reason: before it existed, everyone believed they knew the answers. When I wrote them down and circulated the table, about a third were disputed. Two people each thought the other approved template changes. Nobody thought they owned domain changes. What the document actually did was show where accountability didn’t exist, and force someone to decide.

An agent adds a new question to that table: may an agent make this kind of change, and if so, under whose authority? An agent making a content change to one site under the name of the person who requested it is an ordinary tool, like a text editor that types faster. The template row is harder. Suppose an agent is asked to fix a spacing problem on one operator’s page. It correctly traces the problem to the shared template and fixes it there. The spacing is now correct on a thousand sites, along with whatever else the edit changed. Nobody approved a template change, because nobody asked for one. The agent did better technical work than someone who patched the single page would have, but it also made a change that required a named approver, and that cell in the table is empty.

Consent and liability

That change skipped two questions and left a third for later. The first is authority to act: may this person or agent edit the template at all? That’s a permission, and permissions can be granted to almost anything; a deploy key has one. The second is authorization to accept risk on behalf of the people affected. The template effectively serves a thousand operators and the organization whose name is on the domain. Approval records that someone with standing to consent on their behalf has accepted the risk. That standing comes from a contract, an org chart, or an established working relationship, and it belongs to a person because the affected parties need someone they can hold to the decision. An agent can be given the permission in a minute. I don’t know of any process that gives it the relationship.

The third question is who bears the cost if something goes wrong. An organization can delegate authority to act without delegating authority to accept risk, and it can delegate both without transferring liability. Where the financial cost lands is a matter for contracts, insurance, and lawyers. The operational question is simpler: someone has to decide what happens next and answer for that decision, and the documents should name who that is before it’s needed. In practice that’s often more than one person: someone who can authorize the risk, someone responsible for the system boundaries, and someone who responds when those boundaries fail.

Here’s the case the table doesn’t settle. Suppose the rules are careful: an agent may make reversible content changes to individual sites and nothing else. It replaces an image on one operator’s page, within scope. But the image is also used by the shared template, a dependency nobody fully understood on an inherited platform. A thousand sites show a broken image for the twenty minutes it takes someone to notice and roll back the change. The rollback works, but nothing brings back those twenty minutes, or the screenshot a prospect took in minute twelve. Who owned that risk? It would be convenient to say the person who set the scope. But they believed a single-site change would affect only one site. The scope was only as reliable as the classification underneath it, and the classification was wrong. Blaming whoever wrote the rule wouldn’t solve anything; it would just give the failure a name. The organization needs to decide who is responsible for classifying changes, who maintains the map of shared dependencies, and who authorizes autonomous access on the strength of that map. If those are different people, the handoff between them has to be explicit before the agent is deployed. The agent did what it was allowed to do, and a more capable agent would have done exactly the same thing.

The objection

The obvious objection is that accountability has simply moved up the chain: the agent can’t own the change, so the person who deployed it does. That’s how tools have always worked; nobody holds a compiler accountable. I agree with all of that. My concern is the assumption that comes with it: that naming a person settles how much responsibility that person can meaningfully carry.

Before agents, the number of production changes an organization could make was limited, in part, by the number of people able to make them. Those people could also own their work, so the number of changes and the number of people responsible for them stayed fairly close. Agents loosen that constraint. One person can now authorize a system that makes far more changes than they could ever review. The number of outputs grows; the number of people able to take responsibility for them doesn’t grow at the same rate. Capability scales. Accountability doesn’t.

Suppose you do everything right. Every agent has a named owner, the table is complete, and scopes are tight. The owner of the single-site content scope may still be responsible for hundreds of changes a day across a thousand sites. In the old release process, approval meant someone had looked at the change. At that volume, nobody has looked at every change. So what does the owner actually own? I think the owner owns the conditions the agent operates under: the boundary of the scope, evidence that the boundary is holding, a clear point where the system must stop, and a recovery path for when it fails. The agent’s actions need to be bounded, reversible where possible, logged, and classified against the dependencies the platform really has, which aren’t always the ones anyone wrote down. That’s closer to running a system you can’t inspect one action at a time, which is a different job from the one the release process was written for. I’d say that plainly in the documents. Otherwise the organization keeps the old accountability model on paper while quietly removing the conditions that made it meaningful.

A more sweeping objection is that a capable enough system will make the organization itself unnecessary: no release process because the system doesn’t make mistakes, no owner because there’s nobody left to answer to. I don’t find this persuasive, and my reason has nothing to do with how capable systems might become. The operator whose page is down is still there. So is the prospect, and so is the contract requiring the organization to keep the page up. Even if failures became vanishingly rare, someone would still have to decide what the organization promises, which risks it accepts, and what happens when a promise isn’t kept. Automating the work doesn’t make those obligations disappear, and they’re a separate question from intelligence.

What the redesign looks like

Running a platform that had to survive without its author forced me to think about these questions before any agent arrived. I’d start with three principles, written into the same documents that already govern production work.

First, classify changes by blast radius and reversibility, regardless of who makes them. A change to one site’s content that preserves what it replaces may be safe for an agent to make under the name of whoever requested it. A change to the shared template is a different class and needs an identified owner and approver. The classification has to come from the platform’s real dependencies. The image swap in the earlier example looked like a one-site change. It wasn’t. If the system can’t reliably tell a local change from a shared one, it isn’t ready to grant autonomous permission for that class of change, and hoping the agent reasons its way through the problem won’t make it ready.

Second, make changes reversible by default. Every fix in a site-audit plugin I build saves what it replaces, and I hold the platform to the same rule because it changes what approval means: approving a change that can be undone in a minute is different from approving one that can’t. Reversibility lets an owner authorize more low-risk changes without reviewing each one, provided the rollback is tested and someone can see when it’s needed. It has limits. The twenty minutes still happened, and some consequences can’t be rolled back. What it does is cap how long a failure lasts and give the owner a recovery path they can describe before anything breaks.

The third principle is about scope. A common pattern is to let an agent do whatever its operator can do, under that person’s login, so every change carries the authority of whoever happened to be using it. That’s the compiler model from earlier, and it holds for a compiler because its behavior for a given input is narrow enough that the operator can fairly be said to intend the result. An agent chooses among actions across situations nobody listed in advance. That flexibility is the point, but it means the operator can’t be presumed to intend everything it does. The person who asked for a spacing fix didn’t ask for a template change, and the agent made one anyway. So every agent needs a defined scope with a named owner, and the scope has to be enforceable. It should block actions outside the permitted class and record what the agent did. It also has to say when uncertainty means stop and ask. Then it has to be tested. Before an agent gets autonomous permission for a class of change, someone should have watched it get blocked when it tried to step outside that class, and watched a rollback work on a realistic failure.

None of this needs sophisticated documents. Mine were checklists. Their value was that they existed, that people argued over them, and that the arguments ended in an actual decision about who could do what. Filling in every cell for the sake of completeness misses the point; a name beside a permission isn’t a control if that person has no way to understand or limit the risk. When an agent arrives, these documents are among the few places an organization has written down who owns what. Without them, the agent gets deployed into the gaps, the same gaps that turn a small website problem into a Friday-afternoon crisis.

This is human work. The people who write release processes almost never appear in the AGI conversation about the future of work. They should be at the center of it. If you want to know how quickly an organization can adopt AI, ask to see its release process. Then ask whether it knows which changes an agent can make, how those limits are enforced, and who has to act when they fail.

Friday

I’ll put the argument down and go back to the page.

It got fixed. It usually does. Once I was looking at the right thing, the fix took a few minutes. What I remember is the ten minutes before that: working out whether I was the right person to intervene, whether I’d approved the change that broke it, and what I was allowed to tell the operator in the meantime. An agent could have made the fix in the time it took me to find my laptop. It couldn’t have shortened those ten minutes.

Those ten minutes are the job, and there are about to be many more of them per person. The phone still rings at 4:52, and the person on the other end still wants a name.

Restyle this whole site