How to build a knowledge base for AI customer service
By Caden Boatright
The most common mistake teams make when deploying AI customer service is underinvesting in the knowledge base. They expect the model to figure things out from the product itself — to infer answers from general training rather than specific grounding. It doesn't work.
The model is only as good as the context it retrieves. A poorly maintained KB produces confident, plausible-sounding wrong answers. A well-maintained one makes the agent genuinely useful.
What belongs in the KB
Help center content
Your existing help center articles are the right starting point. They're written for customers, so the language matches what customers ask. Prioritize: FAQ articles, how-to guides, policy pages (return policy, SLA, billing terms), and product feature documentation.
Changelog and release notes
If you ship frequently, outdated answers about features are a major failure mode. Include your changelog. An agent that says “that feature isn't available” for something you shipped three months ago is damaging.
Common ticket resolutions
If you have a backlog of closed tickets with clear resolutions, a curated subset of those makes excellent KB content. Turn the top 20 ticket types into explicit Q&A articles. These are often more useful than formal documentation because they're written in the exact language customers use.
Product specs and pricing
Current, accurate specs and pricing. If your pricing changes frequently, make sure the KB update process is part of the release process — not an afterthought.
What doesn't belong in the KB
- Internal documentation: Runbooks, internal policies, and engineering docs should not be in the customer-facing KB. The risk of the agent surfacing internal information to customers is real.
- Outdated pricing and feature descriptions: Wrong answers about pricing are worse than “I don't know.” Audit and remove stale content.
- Excessive boilerplate: Legal disclaimers and marketing copy dilute retrieval quality. The KB should be dense with specific, answerable facts.
- Unstructured large documents: A 50-page PDF uploaded as-is will chunk poorly. Extract the relevant sections and reformat them as discrete articles.
Chunking and structure
AI retrieval works by breaking your content into chunks and retrieving the most relevant ones. A few principles that improve retrieval quality:
- One concept per chunk: Chunks that mix multiple unrelated topics confuse retrieval. Each article section should cover one answerable question.
- Include the question in the answer: “How do I cancel my subscription? To cancel your subscription...” is easier to retrieve than an article that only contains the answer without restating the question.
- Keep chunks at 300–600 words: Larger chunks include too much noise; smaller ones lose context. Most help center article sections fall naturally in this range.
- Structured headings: Clear H2/H3 headings help chunking algorithms make clean splits.
Maintenance
The KB degrades over time if you don't maintain it. Build a process:
- Update on product changes: Any product update that affects customer behavior (new feature, changed pricing, updated policy) should trigger a KB update on the same day.
- Review on high escalation categories: When you see the AI escalating frequently on a specific topic, that's usually a KB gap. Investigate the escalated conversations and add content.
- Quarterly audit: Once a quarter, review your top 20 ticket categories and confirm the KB has accurate, current content for each.
The work of maintaining a knowledge base isn't glamorous, but it's the highest-leverage investment you can make in AI customer service quality. The retrieval system can only be as good as what it retrieves.
Numbers in this post that cite no source are rules of thumb, not Velor measurements. Velor has no customer deployment to measure yet.