How to Prepare a Knowledge Base for an AI Agent

To prepare a knowledge base for an AI agent, audit the content you already have, remove anything the agent should never see, rewrite each article so it answers a single question in plain language, fill the gaps customers actually ask about, assign an owner and a verified date to every article, and test the agent against a fixed set of real questions before launch. The work is editorial rather than technical, and it determines whether the agent resolves conversations or improvises its way through them.

An AI agent answers best from the content it can retrieve, so the knowledge base sets the ceiling on everything the agent does.

If the AI agent doesn’t have enough context, it may not perform to your expectations. A missing article produces a hedge or an escalation. An outdated article produces a confident answer about a price that changed two quarters ago.

A help center written for people, with context built up across a page and procedures referenced rather than spelled out, produces answers that are fluent, plausible, and wrong often enough to undermine trust in the whole deployment. That pattern is one of the reasons AI-first service becomes a liability in regulated industries, and it is avoidable.

Comm100’s 2026 AI Live Chat Benchmark Report shows where that leaves most teams today.

The distance between handling a conversation and resolving it is almost entirely a content gap and closing that gap is what the eight steps below are designed to do.

What is an AI Agent Knowledge Base?

An AI agent knowledge base is the set of articles, policies, procedures, and reference material an AI agent retrieves from before it answers a customer. In most organizations it overlaps heavily with the public help center and often draws on internal documentation as well. What makes it different from the help center is not the content itself but how that content has been prepared, because the reader is a retrieval system rather than a person.

A person reading a help article skips the preamble, fills in a phrase like "follow the usual refund process" from memory, and notices when two articles contradict each other. A retrieval system does none of those things.

It breaks each article into passages, short sections of a few sentences or paragraphs that are indexed and retrieved independently of the rest of the article. It then scores every passage in the knowledge base against the customer’s question, takes the highest-scoring matches, and hands them to a language model which writes an answer from exactly what it was given and nothing more. If the preamble scores highest, the agent answers from the preamble. If the usual refund process is never spelled out, the agent invents one that sounds plausible.

Throughout this article, "AI agent" means the software, like the Comm100 AI Agent.

How an AI Agent Uses Your Knowledge Base

Nearly every AI agent in customer support runs on retrieval-augmented generation, usually shortened to RAG, and the mechanics are straightforward even if the implementations vary.

A customer asks a question, the system retrieves the passages from the knowledge base that best match it, and the language model writes an answer grounded in those passages. The same retrieval layer sits underneath the different types of AI agents used in support, whether the agent only answers questions or also takes actions, which is part of what separates an agent from a chatbot.

AWS’s guidance on writing for RAG applications covers the formatting mechanics in depth. For anyone deciding what to fix first, we believe that three consequences of the architecture matter more than the rest.

The first is that retrieval works at the passage level rather than the article level, so a paragraph that only makes sense after the two before it will be served to the model without them. The second is that the opening passage of an article is the one most likely to be retrieved, because titles and first paragraphs carry the strongest signal about what an article covers, which means an answer buried in paragraph six may never be seen at all.

The third is that anything the agent can read, it can repeat, including internal notes and comments that were never meant for a customer.

Those three facts turn what looks like editorial housekeeping into something closer to system design. Each of the eight steps that follow addresses one or more of them, and they are ordered the way the work runs in practice: understand what exists, remove what should not be there, fix what remains, add what is missing, then set up the processes that keep it accurate after launch.

Step 1: Audit the Content You Already Have

The first step is an inventory of every source the agent could read, because nothing can be fixed or excluded until it has been found.

That list is usually longer than teams expect. Beyond the public help center there is often an internal wiki, a folder of PDFs, a library of canned responses and macros, policy documents owned by legal or compliance, and release notes maintained by product. Any of those can end up in the index once the agent is connected, so all of them belong in the audit even if some will later be left out.

Teams planning an AI chatbot deployment on an enterprise timeline usually find the inventory is the longest single task in the content workstream, which is a reason to start it first.

For each item in the inventory, record four fields, each of which exposes a different kind of problem.

The owner field tends to generate the most useful conversations, because it surfaces articles written by someone who has since changed roles or left, and those articles are the ones most likely to describe a product that no longer exists.

The audience field surfaces the internal material that becomes important in the next step. Both fields are referenced again later, which is why they are captured now rather than reconstructed under pressure before launch.

Validating the inventory goes faster when the people who know each area do the checking. A product manager will spot a deprecated setting in seconds that a knowledge manager would read straight past, and a compliance lead will recognize an outdated policy clause the same way. Asking each subject matter expert to review only their own area keeps the request small enough to be completed within a sprint rather than sitting in a queue.

Once the inventory exists, run a check that can be called the five-question test. Take the five questions customers ask most often, usually drawn from ticket volume, and search the knowledge base for each one the way a retrieval system would.

For every question, count how many articles could plausibly serve as the answer. A count of one means retrieval has a clear target. A count of two or more means retrieval will make an editorial choice nobody made deliberately, and in practice it often selects the older article because the older article has had longer to accumulate the phrasing customers use. Every question that returns more than one candidate goes onto a merge list, and that list becomes the first piece of work in Step 3.

Step 2: Decide What the Knowledge Base Should Not Contain

With the inventory in hand, the next decision is what to keep out of the index, and it comes before any rewriting because removing content is faster than improving it and prevents the most damaging failures. Three categories of content should never be retrievable by the agent, and each one fails in a different way when it slips through.

The first category is behavioral instruction. Tone guidelines, escalation triggers, guardrails, and anything else the agent needs on every turn belong in the agent’s configuration rather than in the knowledge base.

The reason comes back to how retrieval works. The knowledge base is searched on demand, so content there only surfaces when a customer’s question happens to match it, while configuration applies to every conversation regardless of the question.

Storing a tone guideline as an article causes two problems at once: the agent ignores it on most turns because nothing in the question matches it, and on the rare turn where it does match, the agent retrieves the tone guideline in place of the refund policy the customer actually needed. The knowledge base should hold the detailed, changing facts that vary by situation, such as policies, procedures, and regional differences, and nothing that applies universally.

The second category is confidential material, which is where the audience field from the audit pays off. Not everything internal belongs in this category: non-public knowledge that has been approved for use in customer-facing answers, such as detailed troubleshooting steps or eligibility rules, can safely be given to the agent. Sensitive content cannot. Pricing exceptions, agent-side workarounds, notes about difficult customers, and anything flagged "do not share" need to be excluded entirely or blocked at the content system’s permission layer so the agent never indexes them in the first place.

A prompt instruction telling the agent not to reveal confidential information is not a substitute, and any security team reviewing the deployment will say the same thing. A language model that has retrieved a passage can be persuaded to repeat it, so the only reliable control is ensuring the passage is never retrieved.

Access rules belong in the content system, enforced through roles and single sign-on, not in a request to the model. The security and compliance checklist for a financial services chatbot covers the same principle from the platform side.

The third category is marketing copy and company narrative. Support articles that open with a paragraph about the company’s mission or the product’s award history are common, and they are harmless to a human reader who scrolls past them.

To a retrieval system, that paragraph is a candidate answer like any other, and because it tends to be dense with product names, it can score well against questions it has nothing to do with. Stripping narrative from support content does not make the content colder. It makes the facts easier to find.

One category gets added rather than removed, and it is the one most teams overlook: documentation of what the organization does not offer. If customers regularly ask about a region that is not served, a plan that is not sold, or a feature that is on the roadmap but not yet shipped, that fact belongs in a dedicated article.

An agent with no article on a subject does not stay silent. It retrieves whatever scores highest, which is usually something adjacent, and builds an answer from it. An agent with an article that says plainly "this is not currently available" says exactly that, and the customer gets a correct answer instead of a hopeful one.

Step 3: Restructure Articles so Each One Answers a Single Question

With the exclusions settled, attention turns to the content that stays, and this is the step with the largest effect on accuracy because it aligns the shape of the articles with the shape of retrieval. The merge list from the five-question test is the natural starting point, since every duplicate on it represents a question where retrieval is currently guessing.

The principle underneath everything in this step is that one article should answer one question.

Long omnibus pages, the kind that cover installation, configuration, troubleshooting, and billing for a single product in one scrolling document, perform badly because the platform splits them into chunks for indexing, and a chunk that begins halfway through a procedure has lost the context that made the procedure make sense. A fee listed in one chunk can end up attached to the wrong service in the agent’s answer. Articles should run to a few hundred words, stretching to around 1,500 for a complex procedure, and anything longer should be split along the lines of the questions it answers.

The practical rules follow from that principle.

Scope labels deserve particular attention because they are the rule teams most often skip. A return policy that differs by state needs the state in the title and in the opening sentence, not in a footnote. Without that, the agent applies the right policy to the wrong customer, and it does so fluently enough that neither the customer nor the reviewing human agent notices until the complaint arrives.

The same applies to anything that varies by plan tier, product version, or region. The test is simple: if the first sentence of the article were the only sentence retrieved, would the agent know who it applies to?

When duplicates are resolved, the article with the newer verified date wins, and the loser is archived rather than unlisted. The distinction matters because an unlisted article is hidden from help center navigation but is often still present in the index, which means it remains a candidate for retrieval. Archiving removes it from both.

Content restructured this way tends to serve human readers better as well, because concise, scannable articles with the answer at the top are easier for people to use for exactly the reasons they are easier for retrieval to match. Preparation for the agent and preparation for the help center turn out to be the same work.

Step 4: Write Every Paragraph so It Makes Sense on Its Own

Restructuring gets the shape right at the article level. The next layer down is the paragraph, and it needs its own attention because retrieval does not respect article boundaries. Any paragraph can be served alone, so every paragraph has to carry enough context to be understood alone. That sounds like a minor stylistic point, but it is where most of the subtle wrong answers come from, and it is the part of the work that no platform setting can do on an editor’s behalf.

The rules are few and mechanical enough to turn into an editing checklist.

Repeat the product or feature name rather than writing "it" or "this setting," because the agent may never see the sentence that defined the pronoun. Avoid opening a paragraph with a transition that depends on the previous one, so "alternatively" and "once that is done" give way to a restatement of what was done. Answer yes-or-no questions in full sentences, since a retrieved "Yes" on its own tells the model nothing about what was agreed to, while "Yes, Comm100 Live Chat supports co-browsing on the Enterprise plan" is a complete answer.

Spell out procedures instead of referring to them, because "follow the standard escalation process" is a phrase a human agent decodes from experience, and the AI agent has no experience to draw on. Write firm rules rather than hedged ones, so "a fee may apply" becomes "a $25 fee applies to cancellations made within 24 hours of the appointment." And use the words customers use, pulling the top search terms and ticket subjects into headings and body text rather than relying on the internal product name alone.

Each of those rules addresses the same underlying failure, which is a passage that assumes the reader has already seen something else. Once an editor has internalized that single idea, the rules stop feeling like a list and start feeling like one habit.

The last point in this step concerns media rather than prose. Nothing important should live only in a screenshot, a video, or a PDF table, because retrieval reads text and only text. A pricing table embedded as an image is invisible to the agent, and so is a troubleshooting flow recorded as a video walkthrough. Every visual that carries information needs a text equivalent beneath it, which has the side benefit of making the content accessible to screen readers and to search engines at the same time.

Step 5: Fill the Gaps Customers Actually Ask About

The first four steps improve the content that exists. This step addresses the content that does not, and it comes after the others because filling gaps in a knowledge base still full of duplicates and buried answers only adds to the noise. The table of contents a team inherits usually reflects what someone once thought was worth documenting. The gaps reflect what customers actually need, and the two lists overlap less than most teams expect.

Finding the gaps means starting from demand rather than from the existing structure, and the demand signals are already in the systems most support teams run. The no-result searches from the help center, the top ticket drivers from the last quarter, the chat transcripts where a human agent answered a question with no matching article, the escalation reasons logged in the ticketing system, and the questions human agents answer repeatedly in the inbox each catch gaps the others miss.

No-result searches show what customers looked for and could not find. Transcripts show what they asked a person because they had already given up on finding it.

Once the signals are collected, group them by topic, rank the groups by frequency, and write the top clusters first. Frequency matters because the distribution is steep: a small number of question types usually accounts for most of the volume, and those are where new articles pay back fastest. In iGaming support, for example, a handful of questions about withdrawals, verification, and bonus terms can account for the bulk of incoming volume.

The long tail still matters, though, and edge cases deserve deliberate time after the top clusters are covered. Accessibility requests, what happens to a subscription canceled mid-cycle, what a failed payment triggers, and similar questions are rarely documented because nobody thought to document them, and they are exactly where an agent hallucinates, because there is nothing to retrieve and the model fills the silence.

For teams running Comm100, AI Knowledge identifies gaps in the knowledge base from unanswered or poorly rated interactions and drafts revision recommendations and new articles from those conversations. The output still needs a human owner to review and approve what gets published, which is why ownership is the subject of the next step rather than an afterthought.

Step 6: Set Ownership and Dates on Every Article

Everything up to this point produces a knowledge base that is accurate on launch day. Keeping it accurate afterwards is a process problem rather than a content problem, and the process starts with ownership.

Stale content is worse than missing content, because a missing article produces a hedge or an escalation while a stale article produces a confident answer about a price that changed two quarters ago. The owner field from the audit now becomes an operating requirement rather than a record.

Every customer-facing article needs four things attached to it, and an article lacking any of them is not yet ready for the index.

The distinction between a verified date and an edited date is worth insisting on. An article can be edited to fix a typo without anyone checking whether its substance is still correct, and if the edit resets the date, the article looks fresh while describing a product that no longer exists. Verification means a named person has confirmed the content is accurate as of that date, and only verification should move the date forward.

The cadence only works if content updates are built into the release process rather than bolted on afterwards. The article should change on the day the feature, price, or policy changes, not the week after the first wrong answer appears in a transcript.

The lightest process that reliably holds is a monthly changelog meeting between product, support, and the knowledge owner, where every shipped change is matched against the articles it affects and every affected article is assigned for update. Teams that have been through an enterprise AI chatbot deployment tend to find that this meeting does more for accuracy past the third month than any platform setting, because it is the only mechanism that catches drift before customers do.

Step 7: Test the Agent Before Customers Do

With the content prepared and owned, the agent should be tested against it before any customer is, and the most reliable method is a fixed set of questions known as a golden question set.

The name is borrowed from software testing, where a golden set is a collection of inputs with known correct outputs that a system is checked against after every change. Applied to an AI agent, it is a list of 30 to 50 real customer questions, each paired with the answer expected and the article that answer should come from. Building it takes an afternoon, and it becomes the regression test for every content update that follows.

The questions need to cover more than the easy cases, so they should be drawn from four types in roughly equal measure.

When the set is run, the citations need checking as carefully as the answers, because a correct answer drawn from the wrong article is a failure waiting to happen. The agent may have found the right fact in an outdated duplicate, in which case the answer will break the next time that duplicate drifts further from reality. Every mismatch between the expected source and the actual source goes back to Step 3 as a merge or a scope fix.

When an answer is simply wrong, the fix belongs in the passage that was retrieved, not in the prompt. Prompt patches accumulate silently, they are invisible to the human agents and customers who read the same article, and they treat the symptom while the cause stays in the index.

The out-of-scope questions deserve deliberate attention because they test the path most teams forget to design. Before launch, decide which topics the agent should never attempt, a list that usually includes complaints, legal questions, and anything requiring account authentication. Then confirm that the agent recognizes those topics, says so plainly, and hands the conversation to a human with the context intact.

A well-prepared handoff is a feature rather than a failure, and the benchmark data supports that. In Comm100’s 2026 AI Live Chat Benchmark Report, conversations transferred from the AI agent to a human scored 92.6% on CSAT, up from 86.7% the previous year. Customers do not resent being transferred. They resent repeating themselves, which is why the transcript and the retrieved article should travel into the ticket or live chat the human agent picks up, so the conversation continues rather than restarts.

The Comm100 AI Live Chat Benchmark Report 2026

Compare AI handling, resolution, and handoff satisfaction benchmarks as you test your AI Agent before launch.

Get the report

Report

Step 8: Keep it Current After Launch

Launch is the point where the knowledge base starts generating its own feedback, and the first thirty days decide whether that feedback improves the content or gets ignored until the content decays.

Transcripts need weekly review during that first month, followed by a cadence that matches the volume. They are the richest source of truth a team will ever have about what customers ask and how the agent answers, and no amount of pre-launch preparation substitutes for reading them.

Resolution rate by topic matters because an aggregate figure hides the three topics dragging it down. Escalation reasons matter because a spike in one reason usually points to one article. The questions the agent could not answer are each a gap from Step 5 that surfaced after launch. And articles that are retrieved often but rated poorly deserve the most urgent attention, because an article that is frequently served and frequently wrong does more damage than one that is never found at all.

The loop closes when every correction a human agent makes goes back into the article, so the article wins the next time the question is asked. Frontline agents spot gaps before anyone else, and the reporting path has to be short enough that they use it, ideally a single action from inside the conversation.

On Comm100, AI Copilot draws on the same knowledge base when it suggests replies to human agents, which means one article fix improves the automated channel and the assisted channel at the same time, and the human agents become a second set of eyes on the content without any extra process.

The measure chosen as the target shapes everything above it, and the wrong choice is easy to make. Deflection as a target rewards an agent that answers badly but keeps customers away from people, which is the opposite of what the knowledge base was prepared to do. The benchmark report’s 5.8% reduction in human agent workload is worth having only when resolution rate and CSAT hold alongside it, and a knowledge base maintained against resolution rather than deflection will deliver all three.

Preparing a Knowledge Base in Regulated Industries

The eight steps apply to any organization, but regulated industries add constraints of their own, and Comm100’s 2026 AI Live Chat Benchmark Report shows what careful preparation is worth in each. The common thread is that the cost of a wrong answer is higher, so the separation between what the agent may say and what it may not has to be drawn more sharply in the content itself.

The four verticals below illustrate the pattern where data privacy rules decide what the knowledge base can hold before anyone decides what it should.

iGaming

In iGaming, the benchmark report records a 38.1% resolution rate, the lowest of any vertical in the report, and the gap between that figure and the 75% range seen in education and banking is a content problem before it is a technology problem.

Player questions cluster around withdrawals, account verification, bonus terms, and responsible gambling limits, and every one of those topics carries conditions that vary by jurisdiction, by player tier, and by the promotion running that week.

An article that states a withdrawal timeframe without stating the payment method, the verification status, and the market it applies to will be retrieved for players it does not cover, and the agent will answer them confidently.

The preparation that moves the number is scope labeling and expiry flagging applied with more discipline than most other industries need. Bonus terms expire, so every article describing one needs an end date and an owner who retires it. Verification rules differ by regulator, so articles need the market in the title.

And anything touching a player’s balance, limits, or self-exclusion status belongs behind authentication rather than in the knowledge base at all. Red Chip Media automated 60% of player inquiries on Telegram once its content was prepared along those lines, and the operators at the top of the iGaming distribution in the benchmark share the same pattern: a smaller knowledge base, tightly scoped, maintained against the promotional calendar rather than a quarterly review.

Higher Education

In universities, departments own their own content, and a student’s question routinely crosses departmental lines. A question about a late registration fee touches the registrar, financial aid, and sometimes the bursar, and if each department’s articles do not state which department they belong to, the agent cannot tell which answer applies.

Scope labels and routing matter more than article count here. An AI agent serving dozens of departments can only answer accurately when each department’s content is clearly scoped and routed to the right team. The pressure on that structure peaks during enrollment season, when volume spikes and wait times are the first thing to break.

Credit Unions and Banks

In banking, the distinction that matters most is between general answers and account-specific ones. "How do I dispute a transaction?" belongs in the knowledge base. "What is the status of my dispute?" requires authentication and does not belong there at all, because any article that tries to answer it will be wrong for every customer except one.

The institutions at the top of the banking distribution drew that line early and enforced it in the content, not just in the platform. Most of the AI agent features banks and credit unions need follow from that one separation, and the security and compliance checklist that applies to selecting a platform applies just as much to the content it reads.

State and Local Government

In government, content changes by jurisdiction, plain-language rules apply, and a wrong answer about a benefit deadline has consequences for a resident that a wrong answer about a software setting does not.

Government recorded the highest handling rate in the benchmark because citizen questions are repetitive and suit documented answers, and the 26-point gap between handling and resolving is almost entirely content that exists somewhere in the organization but has not been prepared for retrieval.

That gap is a large part of why AI-first service can become a liability in regulated settings, and why preparation rather than platform choice is where the risk is actually managed.

The Bottom Line

An AI agent cannot resolve what it cannot retrieve, and so much of the success of a good deployment hinges almost entirely on the content behind the agent. Every step in this article, from the audit through to the question set, exists to close that gap.

The knowledge base is the one part of an AI deployment a support team controls completely. Models improve on the vendor’s schedule, and so does retrieval. Content improves when someone on the team decides it will, and it sets the ceiling on everything built above it.

Teams that treat preparation as the first phase of the deployment rather than a cleanup task after launch are the ones whose resolution rates climb month over month instead of stalling at the industry average.

Put Your Knowledge Base to Work With AI Agent

See how Comm100 AI Agent works with the content you already have.

Request a demo

Request Demo

Frequently Asked Questions

The knowledge base is the one part of an AI deployment a support team controls completely. Models improve on the vendor’s schedule, and so does retrieval. Content improves when someone on the team decides it will, and it sets the ceiling on everything built above it.