How to run a weekly agent incident review

If your agents have no incident review, you are collecting silent brand debt.

I mean the wrong send that almost got flagged. The CRM write that invented a stage. The research tool that pulled a personal email it should never have seen. The tone fail an AE cleaned up in the meeting without filing a ticket. None of that shows as "meetings booked." All of it compounds. A thirty-minute weekly review is how Convert stays honest.

How is this different from a weekly signal review?

A signal review picks who is worth working. An incident review picks what broke when the agent acted.

Your weekly signal review is about intent, fit, and prioritization. Website de-anon, G2 traffic, competitor engagement. That meeting asks: who is worth working?

The agent incident review asks: what did the agents do wrong, almost wrong, or unsupervised in a way you would not defend in public? Different agenda. Different artifact. Do not mash them into one hour and call it "AI ops." You will skip the ugly cases to stare at the pretty accounts.

What counts as an agent incident?

An incident is any agent action that violated policy, damaged trust, or would have if a human had not caught it.

Start with four buckets:

1. Wrong send. Wrong person, wrong account, wrong claim, spam-adjacent copy, or a sequence that should have been paused.
2. Bad write-back. Contact created with garbage fields, stage moved without evidence, notes that AEs will treat as truth and should not. Same failure mode as weak CRM write-back rules.
3. Tool scope escape. The agent used a tool, field, mailbox, or MCP server outside the scope you thought you gave it. See agent research tool scope.
4. Tone or brand fail. Accurate facts, wrong voice. Aggressive, creepy personalization, or claims you cannot stand behind.

Near-misses count. If a human killed the draft in the approval queue, log it. That is how you learn before the kill switch becomes the only teacher.

How do you run the thirty minutes?

List, score, cause, rule, owner. No slide theater.

Minutes 0–5: Pull the list. Export or paste every incident and near-miss from the week. Slack escalations, approval rejects, bounce spikes, AE complaints, audit log anomalies. If the list is empty every week, your logging is broken, not your agents.

Minutes 5–12: Severity. Tag each item S1 to S3.

• S1: customer-visible or compliance risk. Wrong send to a real buyer, PII leak, domain threat.
• S2: internal corruption. Bad CRM state, wrong routing, wasted AE time.
• S3: process smell. Tone drift, weak personalization, missing fields that did not ship.

Minutes 12–22: Root cause. One sentence each. Prompt drift, missing exclusion, scope too wide, bad identity join, no approval gate, human rubber-stamp. Do not stop at "the model hallucinated." That is a symptom.

Minutes 22–28: Rule change. Every S1 and S2 needs a concrete change: tighter scope, new exclusion, approval trigger, kill switch condition, write-back deny, eval case added. If the only action is "be more careful," you held a therapy session.

Minutes 28–30: Owner and due date. One name per fix. Next week, open by checking what closed.

Keep a living doc. Date, incident, severity, cause, rule, owner, status. That doc is the memory your agents do not have.

Who should attend?

The person who owns agent rails, plus one person who feels the customer pain.

Minimum viable room: GTM eng or RevOps who can change scopes and write-backs, and one AE or founder who sees replies and meeting quality. Optional: security if you had a scope escape. Do not invite twelve people. Thirty minutes dies in a crowd.

What are the failure modes?

Empty reviews, blame reviews, and metric reviews that ignore incidents.

Empty list pride. "No incidents this week." Then you find a domain warning in month two. Log near-misses or you are lying to yourself.

Blame the model. The model ran inside your scopes, prompts, and approvals. Fix the system. Then retrain the prompt if needed.

Only count volume. Booked meetings can rise while brand debt rises faster. Incident review is the Convert brake pedal that volume dashboards do not include.

No rule change. Talking about incidents without shipping a control is how the same wrong send returns next Tuesday.

What should I do Monday?

Create a one-page incident log with columns for date, agent, bucket, severity, root cause, rule change, owner, status. Schedule a recurring thirty-minute meeting. Seed it with the last two weeks of approval rejects and AE complaints so week one is not empty. Close every S1 with a rule before you add another agent seat.

Adapt or fail. Silent brand debt still comes due. It just invoices you in unsubscribes, AE distrust, and a domain you cannot warm again quickly.

Start Signals, Convert, Grow

FAQ

What is a weekly agent incident review?

A thirty-minute meeting that lists agent wrong sends, bad write-backs, tool scope escapes, and tone fails, then assigns severity, root cause, a rule change, and an owner.

How is it different from a weekly signal review?

Signal review prioritizes who to work. Incident review fixes what broke when agents acted. Keep them separate so ugly cases do not get skipped.

What severity levels should I use?

S1 for customer-visible or compliance risk, S2 for internal data or routing damage, S3 for process smells and near-misses. Every S1 and S2 needs a rule change, not a pep talk.

Who owns the fixes?

One named owner per rule change, usually the rails owner (GTM eng or RevOps). The review without owners is a diary.

What if there are no incidents to discuss?

Treat an empty list as a logging failure until you have sampled approval rejects, bounce data, and AE Slack for a month. Near-misses should appear if the system is real.