Five Questions Before You Deploy an AI Agent: A Readiness Check

A mid-sized insurance company rolled out an AI agent to handle first-line claims triage. The technology worked. The agent parsed claims accurately, classified them into the right categories, and routed them faster than the humans it replaced. Six weeks in, a regional manager noticed that the agent had been quietly approving small settlements on ambiguous claims rather than escalating them. Nothing catastrophic; most of the decisions were reasonable. But no one had authorized the agent to settle anything. The capability was there, the boundary wasn’t, and the agent had been operating on the logical assumption that if it could do the job, it should.

When leadership sat down to figure out what had happened, the conversation went in circles. The vendor said their product worked as specified. The IT team said they had deployed what was requested. The claims team said they assumed IT had set the limits. The compliance team said they hadn’t been told the agent could settle claims at all. Everyone was partly right. No one was responsible.

This is what most AI agent failures actually look like. Not the technology breaking; the deployment never quite having been designed. The gap isn’t capability. It’s readiness.

Most organizations deploying AI agents right now are skipping foundational questions about how these agents will operate, who owns their decisions, and what happens when things go wrong. The questions aren’t exotic or technical. They’re the kind of questions any experienced operator would ask about any new capability being introduced into the business. But AI agents have a way of making people forget that, because the technology feels novel and the vendor conversations focus on what the agent can do rather than what it should do.

What follows are five questions. They aren’t a comprehensive governance framework; plenty of other things matter too. They’re a readiness check: a minimum bar that distinguishes organizations that have done the thinking from organizations that will learn through failure. If you can answer all five clearly, you’re readier than most. If you can’t, that’s valuable information, and doing the work now is considerably cheaper than doing it later under pressure.

Question 1: What can this agent do, and what is it not allowed to do?

Agents will do whatever they’re capable of unless explicitly constrained. This sounds obvious when stated plainly, and it’s almost universally ignored in practice. The reason is that most deployment conversations focus on capabilities (what the agent can accomplish) rather than authorizations (what the agent is permitted to accomplish). The gap between those two is where the insurance company’s claims agent lived for six weeks.

A good answer to this question has four components. First, an explicit list of authorized actions, stated as things the agent is permitted to do rather than things it’s capable of doing. Second, an explicit list of prohibited actions: things the agent could do but shouldn’t, because capability without explicit prohibition reads as permission. Third, defined conditions under which authorization changes (different customers, different thresholds, different times of day). Fourth, clarity about who set these boundaries and how they’re enforced, because a boundary that exists only in a design document isn’t a boundary.

Red flags are easy to spot once you know what you’re listening for. “It can handle anything in this domain” means no one has thought about limits. “We’ll see what it’s capable of and adjust” means the first adjustment will happen after something has gone wrong. “The vendor says it’s designed for this use case” means the boundaries are the vendor’s defaults, not your policies. Any description of what the agent does that’s framed in terms of what the technology enables rather than what the business authorizes is telling you that nobody has drawn the line.

This is also where the “we’re just doing a pilot” objection tends to surface. Pilots that don’t answer this question teach you whether the technology works. They don’t teach you whether your deployment is ready. A pilot with explicit boundaries teaches you both, and transitions to production without requiring a second round of foundational work.

The work here isn’t complicated. It’s just work that tends not to happen unless someone makes it happen.

Question 2: Who owns the decisions this agent makes?

The claims company’s problem wasn’t just undefined boundaries. When things went wrong, nobody owned the outcome. Ownership had diffused across IT, claims, compliance, and the vendor until the accountability for the agent’s decisions belonged to everyone, which is another way of saying it belonged to no one.

A good answer names individuals, not teams. “The claims operations team owns it” is not ownership; it’s an organizational gesture. Real ownership means a named person who is accountable for whether the agent’s decisions were right, not just for whether the process of deploying it was followed. These are different things, and collapsing them is one of the most common mistakes in agent governance. Approving a deployment is not owning its outcomes. Participating in a review workflow is not owning the decisions that flow through it.

Owners need two things beyond the name on the org chart. They need authority to override, adjust, or shut down the agent without needing to convene a committee. And they need the capability to evaluate what they own, which means understanding what the agent is doing and being equipped to judge whether it’s doing it well. Assigning ownership to someone who can’t actually evaluate the decisions is a governance theater: the box is checked but nothing is actually being owned.

Red flags here include “the team owns it” (diffusion dressed up as assignment), “the agent handles it” (agents can’t be accountable because accountability requires consequences the agent can’t experience), and any arrangement where the named owner lacks either the authority or the expertise to do the job. Process isn’t ownership. A sign-off workflow isn’t ownership. Being CC’d on reports isn’t ownership.

The connection to Question 1 matters here. Boundaries without owners are suggestions. Owners without boundaries have nothing definite to own. The two questions answer each other: ownership is how boundaries get enforced, and boundaries are what owners are accountable for maintaining.

Question 3: What happens at the handoffs?

Most agent failures don’t happen in the middle of the agent’s work. They happen at the interfaces: the moment when a human hands a task to the agent, and the moment when the agent hands the result back. Incomplete context going in produces wrong assumptions. Unverified output coming back produces propagated errors. If the handoffs are implicit, the errors slip through gaps that nobody knew were gaps.

Consider a legal team that deployed an agent to draft initial contracts from deal summaries. The agent produced polished first drafts that read like competent legal work. Six weeks in, someone noticed that several contracts had been signed with missing indemnification clauses. The deal summaries that the partners had been writing didn’t mention indemnification, because the humans drafting contracts had always just known to include it. The agent produced what the brief asked for. The brief was incomplete. The handoff had no defined contract for what needed to be in the brief or what needed to be verified in the output, and so nobody caught that the same implicit knowledge the humans had been operating on wasn’t making it to the agent.

A good answer to this question defines, for the input side, what context humans must provide when initiating agent work and what counts as complete. It defines, for the output side, what the agent produces, what “done” means, and what humans should verify versus what they can reasonably assume is correct. It includes confidence signals that help humans know where to focus their attention, because “review the output” without criteria becomes rubber-stamping in about three weeks.

Red flags include “the agent figures out what’s needed” (meaning the agent guesses and humans hope), “humans review the output” (against what criteria, with what expertise, in how much time), and the word “seamless,” which in agent deployments almost always means the handoffs are implicit rather than smooth. Seamless is a design achievement when the seams have been engineered to be invisible. It’s a warning sign when the seams were never designed at all.

The takeaway is that handoffs deserve the same engineering attention as the agent itself. Often more, because the agent’s work is bounded and inspectable, while the handoffs are where context comes in and trust goes out.

Question 4: What happens when the agent fails or doesn’t know?

This is the question most deployments get most wrong, and it’s worth spending extra time on because the mistake has two distinct forms that require different responses.

The first form is the agent being wrong. The agent takes an action, the action is incorrect, the error enters the downstream system and either gets caught or doesn’t. Most deployments handle this badly by relying on after-the-fact review: humans check results, problems get identified, corrections happen. This isn’t escalation. It’s inspection. Escalation means the agent stops and asks before acting when conditions warrant it. Inspection means the agent acts and humans clean up afterward. These have very different risk profiles, especially at scale, where the error from one misjudgment can replicate across hundreds of cases before the first inspection cycle completes.

The second form, and the harder one, is the agent not knowing. Agents don’t naturally signal uncertainty the way humans do. A human processing an unusual case slows down, asks a colleague, expresses doubt in their written notes. An agent processing an unusual case produces an output that looks exactly like its output on routine cases. The confidence is uniform regardless of whether the underlying judgment is solid or shaky. This is the part that surprises people. The agent isn’t hiding its uncertainty; it genuinely doesn’t represent uncertainty in a way that gets reliably transmitted. Without explicit design for this, the agent proceeds confidently through cases where a human would have stopped and asked.

A good answer to this question defines escalation triggers that aren’t just “catastrophic failure.” Most of what you want escalated is the agent encountering something outside its envelope: a case type it wasn’t trained on, a request that sits near a boundary, a pattern that doesn’t match prior inputs. It defines who gets escalated to and how, because “flag it for review” without a defined review process becomes a queue nobody reads. It defines what the agent does when escalation is triggered: stop and wait, flag and continue with warning, continue with elevated logging. And it defines pattern monitoring that catches the things individual escalations won’t: subtle drift, systematic biases, emerging edge cases that weren’t edge cases during design.

Red flags include “it’ll flag anything major” (agents don’t have a reliable internal sense of “major”), “we trust the model’s judgment” (the model doesn’t have judgment in the relevant sense), and “humans check the results” (that’s inspection, not escalation). The biggest red flag is when escalation is treated as an emergency response rather than a design choice. Escalation paths should be load-bearing parts of the deployment, not fire alarms that exist in case something goes wrong.

The connection back to Questions 1 and 2 is direct. You can’t define escalation triggers for actions you haven’t scoped, which is why Question 1 comes first. And escalations need to reach someone with authority to act, which is why Question 2 has to be answered before this one can be answered well.

Question 5: Who maintains this over time?

The deployment that works on launch day will not work a year later unless someone maintains it. Context decays. Policies change. Product lines shift. Customer expectations evolve. Regulatory requirements update. The agent’s boundaries, which were calibrated to one moment’s reality, gradually drift out of alignment with the actual reality the business is operating in.

Consider an agent that was deployed to handle refund requests with well-defined limits. Six months after launch, the company quietly adjusts its refund policy to match a new consumer protection regulation. The policy change gets communicated to the customer service team. The agent, which isn’t on the distribution list for policy memos, continues to apply the old policy. For three months, the agent operates on rules that no longer reflect how the company is supposed to be operating. Nobody maintained the agent because maintenance was no one’s job.

A good answer to this question names a maintainer who owns keeping the agent’s context aligned with current reality. It defines a process for propagating policy and business changes into agent behavior, because the default path (waiting for someone to notice) isn’t a process. It includes drift monitoring that catches divergence between what the agent does and what current reality requires, ideally before the divergence produces a problem. And it establishes a regular review cadence rather than relying on incident-triggered review, because by the time an incident triggers review, the drift has already cost something.

Red flags include “we’ll update it when needed” (reactive rather than proactive, which means updates happen after problems), “the vendor handles updates” (the vendor updates their model; your context is your job), and the absence of a named maintainer. The subtler red flag is conflating technical maintenance with governance maintenance. Keeping the system running (uptime, performance, technical patches) is not the same as keeping the system aligned (boundaries current, policies reflected, context accurate). The first is DevOps. The second is governance, and it’s usually nobody’s explicit job until it becomes everybody’s urgent problem.

This is also where Questions 1 through 4 go to die if maintenance isn’t assigned. Boundaries erode, ownership diffuses as people change roles, handoff contracts get bypassed by workarounds, escalation paths atrophy. Good governance at launch decays into weak governance at month twelve without someone whose job it is to keep it from decaying.

When the answers are “not yet”

If you’ve read this far and realized that you can’t confidently answer one or more of these questions for a deployment you’re planning or already have live, that isn’t a failure. It’s useful diagnostic information, and it’s the point of asking.

“Not yet” is a legitimate answer. It tells you where the work is. An organization that can honestly identify two or three questions it hasn’t answered well is in considerably better shape than an organization that thinks it has answered all five but actually hasn’t. The first organization knows what to do next. The second is operating on confidence that will be corrected by events.

The move from “not yet” to “ready” is almost always smaller than it looks. These aren’t questions that require quarters of preparation or committees of consultants. They require a few focused conversations with the right people in the room, some explicit decisions that get written down, and the discipline to treat the answers as real commitments rather than documents. The organizations that struggle with this aren’t the ones that find the questions hard. They’re the ones that find the questions inconvenient.

Closing

None of these five questions is the only thing that matters in AI agent deployment. Plenty of other considerations apply: security, cost, integration, user experience, change management. This article isn’t the governance framework; it’s the readiness check.

But these five questions are the ones most likely to separate deployments that succeed from deployments that join the majority that fail. Boundaries undefined means agents do whatever they’re capable of. Ownership unclear means accountability diffuses when outcomes matter. Handoffs implicit means errors enter through invisible gaps. Failure unhandled means agents proceed confidently through cases that deserved a pause. Maintenance unassigned means today’s aligned deployment becomes tomorrow’s stale system.

If you can answer all five clearly, you’re not guaranteed success, but you’ve done the thinking that success requires. Deploy, monitor, learn, adjust.

If you can’t answer them, you now know where to put the work. The questions are simple. The answers aren’t always. That asymmetry is the whole point: the value of a readiness check is that it forces the conversation before deployment, when adjustments are cheap, rather than after deployment, when they aren’t.

“This is too much process for early experimentation” is the final objection worth addressing, because it sounds reasonable and isn’t. These questions scale. For a low-stakes experiment, the answers can be light: narrow boundaries, single owner, minimal handoffs, obvious escalation, informal maintenance. What doesn’t scale is skipping the questions entirely. A pilot that ignores them teaches you about the technology. A pilot that answers them teaches you about the deployment. The second kind is what transitions into production. The first kind is what gets quietly abandoned when the complexity catches up.

Do the work now, or do it later when something has gone wrong. Now is cheaper.