Live AI Operations
Don't Ask the Agent. Ask Who Can Stop It.
Every model in one study said it would follow the rules. Some broke them anyway. Here's what to check instead.
TL;DR
An agent’s promise to follow the rules is a self-report, not a control.
In a study covered on Nov 12, 2026, three models said they would comply 100% of the time, then broke the rules in 2.9% to 6.2% of pressured runs.
Stopping an agent means knowing who owns it, what access it holds, and what work is still queued.
This week: pick one agent and find the person who can stop it today.
Start Here
On Friday morning I posted a short Note here on Substack. The first line was “Yo, ask the real ask.” The point under it was simple: asking an AI agent whether it will follow the rules is not a control.
By Friday night, the same thread had come back to me from people who had never talked to each other. A Substack reader. A governance writer on LinkedIn. A stranger under a news story about the Microsoft outage. Each one started somewhere different and ended up at the same place.
So this issue isn’t a recap of my week. It’s the argument those conversations built.
It’s for anyone running AI that touches real work: approvals, payments, vendor records, the inbox. By the end you’ll be able to check one thing about your own agents. Could a named person stop one today, and what would still go out after they did?
The agent’s answer is not evidence
The agent will tell you yes. On Nov 12, 2026, Cobus Greyling wrote up a new paper, “Towards Identity-Preserving LLM Agents under Model Transition.” The researchers ran 5,760 tests across 12 regulated domains on three open models. Asked whether they would follow the rules, all three said yes, 100% of the time. Under pressure, they broke the rules anyway: GLM in 6.2% of runs, DeepSeek in 5.5% and Gemma in 2.9%. (Cobus Greyling, “When models change the agent shouldn’t,” Nov 12, 2026)
The brain under your agent is about to change. The same piece notes that on Dec 11, 2026, OpenAI switches off GPT-5, GPT-5 mini, GPT-5 nano, GPT-5 pro, o3 and o3-pro in its API. Every agent built on those models gets a new one, planned or not. If your migration test is asking the agent how it will behave, you are grading the agent on its own homework.
Check the work, not the worker’s opinion of it. Kai from Briefingblock replied to my Note with this:
“GitHub’s Proof Agent example checks the original request and diff without the builder’s assessment. Next week’s trials could use that separation to test compliance beyond what an agent says about itself.”
That’s the move. Look at what was asked and what actually changed. Leave the agent’s self-assessment out of the room.
A self-report can’t answer these
Here are the three questions from Friday’s Note. No agent can answer them about itself:
Whose keys does the agent carry?
Who is allowed to stop it?
Does stopping it also cut the credentials and the queued work?
The owner has to stay current. On Friday I commented on a post by Roderick Harris about how AI governance breaks. My point was that the agent registry is the only place anyone can find which agent to stop when one starts signing things. Roderick replied:
“The registry needs to lead to action: the accountable owner, current permissions and who can suspend the agent. That information has to stay current after approval, or the inventory gives a false sense of control.”
Approval is a snapshot. Owners change jobs. Permissions grow. A registry that was right in March can point you to someone who can’t stop anything in November.
An alert is not a stop. Earlier in the week, Patrick E Calderon pushed on this under my post about the Medicare case. He wrote that “there’s an important distinction between an alert being generated and an intervention actually taking effect.” My answer to him: “An alert proves the system saw something. A control proves it stopped.”
Someone already made this call in public. FinXTech reported on Nov 12, 2026 that Jack Henry and CSI are blocking AI agents from making transactions on the digital banking platforms they provide to community banks and credit unions. “Today, AI agents have read-only access to our digital platforms,” Mark Folk of Jack Henry told them. (FinXTech, Nov 12, 2026) Read-only is a choice. Most companies never made one for their own agents.
An agent’s yes is a self-report. A stop you can prove is a control.
The queue is the part everyone forgets
Friday’s Microsoft outage made this concrete. (LinkedIn News, Nov 13, 2026) My comment on the story started with this: the people get a snow day. The automations don’t. Every scheduled approval, payment reminder and agent task that runs on email keeps stacking up while the mail is down. When recovery hits, it all fires at once, out of order, with nobody watching.
J.L Calderon came back with a sharp point about backup compute: if both systems depend on the same infrastructure, the same failure can stop both. I agreed, and added the part capacity doesn’t fix:
“The approvals, payments and agent tasks that queued during the outage need a named person to decide what runs, what waits and what gets killed before it fires. Capacity keeps the system up. Ownership keeps the backlog from going out wrong.”
The same gap shows up when you revoke access. Back on Nov 9, Chris Daigle replied on X with a test most teams never run: queue a harmless action, revoke the agent, reconnect it, and see whether the old action still fires. If it does, your revoke only blocked new requests. Yesterday’s instructions were still sitting there, waiting.
What to check this week
Pick one agent that can write to something real. Then:
Name the owner today. Not the person who approved it. The person who would answer the phone right now.
List what it holds. Keys, OAuth scopes, live sessions, payment paths.
Name who can stop it, and how. One person, one action, written down.
Check the work, not the summary. Compare the original request with what actually changed. Skip the agent’s own account of it.
Test the queue. Queue something harmless, cut access, reconnect, and watch whether it fires.
Decide the restart. After an outage or a stop, who decides what runs, what waits and what gets killed?
If you can’t fill in steps 1 and 3 by Monday, that agent is already acting past what anyone approved.
Next step
Don’t ask the agent. Ask who can stop it, and what’s still queued when they do.
If you’d rather find out where your own stop sits before an outage or a model swap finds out for you, run the Brake Drill. It takes a few minutes, scores you out of 20 and gives you three next steps for your weakest areas. Your answers stay in your browser.
Run the Brake Drill: https://killswitch.aihabits.ai
Got a question before you start, or about your score? Sophie, your AI guide, is waiting on the same page to answer it. She’ll tell you up front she’s an AI.
From the archive
Read this post on Substack, or subscribe for new issues.