Skip to main content
Velor
All resources
StrategyApril 2026 · 7 min read

Scoping an AI customer service deployment for the first 90 days


Most AI customer service deployments that struggle in the first quarter don't fail because of the technology. They fail because the team never agreed on what success looks like before they started. This is a framework for the decisions that matter most in the first 90 days — before launch, during the first month, and through the often-surprising middle stretch.

Before you start: define what “resolved” means

This sounds obvious, but almost every team skips it. Before you turn anything on, you need a written definition of resolution that everyone — product, support, and leadership — agrees on.

The default mistake is measuring deflection: the bot handled it without a human touching it. Deflection is easy to measure and feels good on a dashboard. It's also nearly meaningless as a quality signal. A bot that confidently gives wrong answers has a high deflection rate. A bot that escalates to a human agent whenever it's uncertain has a lower one, but is actually doing its job better.

Resolution is harder to measure but more honest. A workable definition: the customer rated the interaction helpful, or they did not open another ticket on the same issue within 48 hours. A customer who came back with the same problem wasn't resolved, regardless of whether a human touched the first conversation. Set this definition before you have any data. Changing it later makes trend lines meaningless.

Month 1: get one channel right

Don't launch chat, email, and voice simultaneously. The failure modes across channels are different, and trying to debug all of them at once means you won't debug any of them well.

Start with the channel where failure is cheapest. Usually that's async chat — a wrong answer there is frustrating but recoverable. The customer can clarify, the agent can correct, and nothing is irreversible. Voice is the opposite: a wrong or stilted answer in a live call ends the relationship.

Before launch on your first channel, pull your top 20 ticket categories from the past 90 days and make sure each one has at least one KB article that directly answers it. Not an article that vaguely covers it — one that answers it. You're not trying to cover everything before launch. You're trying to cover the things that will immediately break if you don't.

Instrument your resolution rate from day one. You need a baseline. Without one, you can't show improvement at day 30, and without that you can't justify expanding scope.

The knowledge base surprise

In month one or two, almost every team discovers the same thing: their internal documentation isn't written for retrieval. It's written for humans who already know the product.

“Update your settings in the dashboard” is a perfectly clear instruction to someone who knows your product. It tells a retrieval system almost nothing useful — which dashboard, what settings, where exactly. The AI either retrieves this article and answers vaguely, or retrieves nothing and escalates or hallucinates. Neither is good.

The fix is a rewrite discipline, not a total rebuild. For each article in your top 20 categories, apply three tests: Does it answer exactly one question completely? Does it include the specific UI text, button labels, and menu paths a user would search for? Does it assume zero prior knowledge of the product? If an article fails any of these, it needs editing before you can trust the retrieval results it produces.

This is the work that feels like documentation cleanup but is actually the highest-leverage technical work in the first 60 days. The AI is only as good as what it can retrieve.

Month 2: add actions carefully

Once your knowledge base is stable and your resolution rate has plateaued on KB-only questions, you're ready to add your first custom action. The most common starting point is a read-only lookup: order status, subscription status, account balance. Something the agent can check and report without changing anything.

Start read-only. Verify that the agent calls the action at the right moments, with the right parameters, before you add anything destructive. An order lookup that fires unnecessarily is a minor irritant. A cancellation that fires unnecessarily is a customer relationship problem and potentially a revenue problem.

The reason HMAC signing and confirmation guardrails exist for write actions is precisely this: write actions that fire incorrectly are hard to reverse. Building in a confirmation step — a card showing the exact action and parameters, confirmed in the chat widget or approved by a teammate on other channels — catches accidental triggers. It also gives you a log of what the agent was about to do, which is useful for debugging misfires.

Month 3: escalation tuning based on data

By month three, you have something you didn't have at launch: conversation logs that show exactly where the agent gets stuck. This is when escalation configuration moves from intuition to evidence.

The patterns that show up most consistently: billing disputes almost always warrant escalation, because the right outcome depends on context a KB article can't fully capture and the stakes of a wrong answer are high. Complex multi-product questions often reveal KB gaps — articles that cover each product individually but not the interaction between them. Emotionally charged tickets — angry language, explicit frustration, mentions of leaving — are better handled by a human regardless of whether the underlying question is answerable by the agent.

Tune your escalation signals based on what you see in the logs, not what you expected at launch. The categories where the agent gets stuck more than 40% of the time are escalation candidates. The categories where it has a high resolution rate can absorb more volume.

What the 90-day milestone actually looks like

At 90 days, you're not celebrating — you're making a decision. The question is whether your resolution rate, your escalation patterns, and your action performance justify expanding scope: more channels, more ticket categories, or higher volume from the same channels.

If you defined resolution correctly at the start, instrumented from day one, and built your KB with retrieval in mind, that decision is an engineering decision — grounded in data. If you skipped any of those steps, it's a guess. The 90-day process is designed to make sure you're not guessing.

Numbers in this post that cite no source are rules of thumb, not Velor measurements. Velor has no customer deployment to measure yet.

Velor

Want to see how this applies to your support queue?

Send us your recent tickets — at least 300, ideally 500 — and get a resolution rate, before you commit to anything.

14-day free trial · no credit card