Skip to main content
Velor
All posts
PerspectiveOctober 2026 · 5 min read

Who decides what counts as resolved? Questions to ask any AI support vendor

By Caden Boatright


“Resolution rate” is now the number every AI support vendor leads with. It is a better number than deflection, but only if you know what is inside it. Two vendors can report the same figure and mean different things, and the difference is usually who defined “resolved” and who did the grading.

If you are comparing vendors, these are the questions worth asking before you sign anything.

1. What has to be true for a conversation to count as resolved?

Get the definition in writing. Does the customer have to confirm? Does a customer who simply stops replying count? What about one who comes back a day later about the same issue? A definition that counts silence as success will always look better than one that does not, and you will never see the difference in the headline figure.

2. Who grades the conversations?

If the vendor's own model decides which conversations were resolved, the vendor is marking its own homework. That is not automatically wrong, but you should know it, and you should be able to read the conversations behind the score. Ask for a sample you can check by hand.

3. Is the number tied to what you pay?

When pricing depends on the count of resolutions, the definition stops being a measurement question and becomes a commercial one. Ask how disputed conversations are handled, and whether you can audit the count for a month before it affects an invoice.

4. Was it measured on tickets like yours?

A rate measured on someone else's ticket mix tells you about their mix. Your tickets, with the full customer message rather than just the subject line, are the only sample that predicts your result. Subject lines are short and tidy, and scoring on them flatters the agent.

5. How much uncertainty is in the number?

A single percentage with no interval hides how much a few hundred tickets can swing. Ask for the confidence interval. If a vendor cannot give you one, the figure is an anecdote.

6. Can you see every miss?

The failures matter more than the average. You want the list of tickets the agent got wrong or should have handed to a person, so you can judge whether they are the kind you can live with. A summary that omits them is a sales deck.

How Velor answers these

Velor's free eval is built around these questions. You send 300 to 500 of your real tickets, with the customer's full message and, where your export carries one, your team's reply, from a Zendesk, Intercom, Gorgias, HubSpot or Freshdesk export as it is. Velor replays each one against your own help center and reports the resolution rate with a 95% confidence interval and every miss listed. Scoring uses the full message body. It takes however long your export and knowledge base need, we sign an NDA first, and names and emails can be stripped before you send anything.

We do not quote a resolution rate on this page, because we have no published customer results and the number depends entirely on your tickets. The eval produces yours.

Run the same test on anyone

Nothing above is specific to Velor. Hand any vendor the same 500 tickets, ask the six questions, and compare what comes back. The vendor that is comfortable being checked is the one worth shortlisting.

To start with Velor, request the free eval. For the scoring approach in more detail, read how to evaluate an AI support agent and why resolution rate beats deflection. If your main worry is the tickets an agent should hand to a person, read escalation design: when the AI should hand off and how.

Numbers in this post that cite no source are rules of thumb, not Velor measurements. Velor has no customer deployment to measure yet.

Velor

Want to see how this applies to your support queue?

Send us your recent tickets — at least 300, ideally 500 — and get a resolution rate, before you commit to anything.

14-day free trial · no credit card