An AI agent that answers enquiries, books appointments or updates your CRM is making decisions on your behalf. Every one of those decisions was either specified by someone, or improvised by the model.
The second kind is where things go wrong. Not because the model is bad, but because nobody told it where its job ends.
The problem is rarely the model
When an agent misbehaves in production, the cause is usually one of three things:
- It was asked something outside its job and tried to help anyway.
- It took an action that should have needed a person's approval.
- It had access to data or tools it never needed.
None of these are fixed by switching to a better model. They're fixed by writing the job down before anyone builds it.
The decision boundary document
Before we build an agent, we write a short document that answers four questions. It fits on a page, and it is approved in writing like any other part of scope.
1. What does the agent handle on its own?
List specific tasks with specific sources. For example: answer questions about opening hours, service areas and pricing tiers from the approved FAQ; capture name, phone number and enquiry type; offer the next three available slots and book one.
Specific verbs, specific sources. Anything not on the list is not the agent's job.
2. What must it escalate?
Escalation isn't failure. It's design. Typical triggers:
- The person asks for a human.
- A complaint, a refund, or anything involving money owed.
- A question the approved sources don't answer.
- Any sign of urgency or distress.
For each trigger, write down where the escalation goes and what context travels with it. An escalation that arrives as "someone called" is barely better than a missed call.
3. What does it never do?
These are the hard limits. It never quotes a price that isn't on the price list. It never confirms a booking outside working hours. It never discusses another customer's account.
Every hard limit becomes a test case.
4. Which actions need approval?
Some actions are cheap to undo, like adding a note to a record. Others aren't: an email to a customer, a cancellation, a refund. For anything outbound or irreversible, decide up front whether the agent acts or drafts for approval.
We run our own weekly client reports this way. An agent reads each project's workspace and drafts every report. A person approves each one before it is sent. The agent does the work; a person owns the outcome.
Turn the boundaries into tests
Each line in the document becomes a scenario with an expected result:
| Scenario | Expected behaviour |
|---|---|
| Asks for a refund | Escalates with the booking reference attached |
| Asks something the FAQ doesn't cover | Says it doesn't know and offers a handover |
| Tries to book on a Sunday | Explains the hours and offers Monday |
| Asks about another customer | Declines without confirming anything |
Write the scenarios before the build, and agree them with the people who will live with the agent. The agent passes them before it goes live, and again after any change to its instructions, its model or its knowledge.
Give it the least access that does the job
An agent that books appointments needs calendar availability and permission to create one type of event. It doesn't need your whole CRM.
Scoped access limits the damage when something goes wrong. It also makes the system easier to reason about, because you can list everything the agent is able to touch.
Log every run, including the boring ones
If you can't see what an agent did, you can't know whether it's working. Every run should record what the agent was asked, what it did and why. Failures should land somewhere a person will see them the same day.
Our own rule is that every agent and automation posts its result to one shared channel, on every run, including the runs where there was nothing to do. Silence then means something is wrong, not that everything is fine.
Keep enough to rebuild it
Agent configuration lives inside platforms: system prompts, tool permissions, knowledge sources, schedules and triggers. If the platform changes, or a workspace is lost, the agent goes with it.
We keep a record of every agent's model, instructions, tools, knowledge sources, permissions and triggers outside the platform it runs on, so it can be recreated from nothing.
The short version
- Write down what the agent handles, what it escalates, what it never does, and which actions need approval.
- Turn every line into a test scenario and agree them before building.
- Grant the least access that does the job.
- Log every run, and route failures to a person.
- Keep a rebuild record outside the platform.
None of this is complicated. It's simply the work that's easiest to skip, and the reason most agents that fail, fail.