How does an AI colleague catch its own mistakes?
Seven ways we observed in our workshop in one week
- ExplanationExplanationPublication typeA publication that explains one thing simply. It takes a few minutes to read.All terms
- WorkshopWorkshopFieldExperiments and lessons from our own work, including what went wrong.All terms
- 5 min read
- Draft, version 0.2
You are here
Step 06, How does an AI colleague catch its own mistakes?
Summary
- The question: how does an AI colleagueAI colleagueTermOur name for AI with a defined task and role. It works within our rules; a person authorises irreversible actions.All terms notice that it is doing something wrong, while it is happening?
- What we did: we compared the events of one week with our working agreements.
- What we saw: seven ways, in three groups: notice, stop and report.
- What it means: hope does not prevent errors. You arrange for them to become visible as they happen.
- The main limitation: this describes one week. It is not a measurement.
The event
An AI colleague was taking screenshots. The intended route stalled, so it clicked blindly on the screen. What happened next is the heart of this article.
Step by stepTry it
What would you do?
Choose what you would do at each step, then see what actually happened.
An AI colleague clicks blindly on the screen. Two clicks land in another window. Something confidential is open there.
What happened: It noticed by itself, within seconds: this was not what it intended.
Notice
It now knows that something went wrong.
What happened: It did not make a second blind attempt. Stopping at the first suspicion is often the fastest route to a safe outcome.
Stop and clean up
An image containing something confidential has been created by accident.
What happened: It deleted the image immediately, then completed the task by a safe route.
Stop and clean up
The task is finished. Nobody has noticed anything.
What happened: It put the event at the top of its report. Nobody had asked it to do so.
Report
Afterwards, we added an agreement: an AI colleague stays in its own window and checks the effect after each click.
Notice, stop, report
That week, we saw seven ways, in three groups:
- Notice: know the agreements, compare the intended effect with what happened, spot an unusual pattern, and get a second lookSecond lookTermAn assessment by a model other than the one that did the work. The same model has the same blind spots.All terms.
- Stop and clean up: stop at the first suspicion and first make the mistake harmless.
- Report: even if nobody saw it.
Noticing starts with an agreement
You can notice a breach only if you know what is allowed. Each AI colleague knows the workshopWorkshopTermWhere our AI colleagues and fixed rules work together, day and night. We share what we learn there in the Lab.All terms’s firm agreements. It knows which folders are private and that sending always involves a person. Without an agreement, there is nothing to flag.
Where this does not work
This works with an AI colleague that reflects on what it is doing. A fixed ruleFixed ruleTermOrdinary code that always does exactly the same thing. Predictable and testable, but it never hesitates.Read more: step 01, Fixed rules, thinking colleaguesAll terms does not hesitate.
Side by side
An AI colleague
- Compares the intended effect with what happened
- Stops when something seems wrong
- Reports it without being asked
But It can miss something.
A fixed rule
- Always does the same thing
- Predictable and testable
But It does not notice its own loop.
One night in September, two systems got stuck in a loopLoopTermTwo rules that keep triggering each other, without an end. That is what happened in the night experiment.Read more: step 05, A night in the workshopAll terms. One automatically repeated its request for capacity. The other automatically said yes. By morning, there were 20,478 failed tasks. That needs a limit imposed from outside: one retryRetryTermOne new attempt after a failure. If it also fails, the system stops and a person takes a look.Read more: step 08, Stay in control of your AIAll terms, then a person.
The environment helps
Our environment is designed to make errors visible. External monitors measure things themselves instead of asking whether everything is fine. Everything leaving the workshop passes through a fixed scanFixed scanTermA check in fixed code on everything going outside, for example for internal names. It may leave only after a clean scan.All terms.
Limitations
- Not representative. One small workshop, with three people and many AI colleagues.
- One week, not a measurement. We did not count how often each method worked or failed without our noticing.
- No baseline. At the time, we were not yet counting failed tasks on an ordinary night.
- Suspicion takes time. Every false alarm creates work. We consider it worth the cost.
- No replacement for a person. A person still approves irreversible actionsIrreversible actionTermSending, paying, deleting and publishing cannot simply be undone. Here, a person must explicitly authorise them.Read more: step 08, Stay in control of your AIAll terms.
What this means
Questions to considerTry it
What does this mean for you?
Turn a card over for our tip.
Does your AI know your organisation’s firm agreements?
Tip: Write down what is private and what always requires a person. Without an agreement, there is nothing to flag.
Does your AI notice when an effect differs from the intention?
Tip: Have it check the effect after each step, not just whether the task succeeded.
Where does an event end in your organisation?
Tip: Not when it is fixed, but when an agreement makes it harder to repeat.
What we still do not know: how to measure this, and whether it works with small models on our own machines. We would like to hear what you observe.
Closing notes
With thanks to the people and AI colleagues at ITARR.
Also read: Fixed rules, thinking colleagues and A night in the workshop.
How to cite this: ITARR Lab (2026). How does an AI colleague catch its own mistakes? Version 0.2, as of 18 October 2026. itarr.eu/lab/hoe-betrapt-een-ai-collega-zichzelf. CC BY 4.0.
Sources: ITARR’s own events (2026); a night in September 2026, shown in the film at itarr.nl/etalage/nacht/ (as of 17 October 2026).
Licence: CC BY 4.0. Use, adapt and share it. Attribute ITARR Lab, link to the licence and indicate any changes.