Source: https://www.svs-digital.com/guides/ai-voice-agent-knowledge-base-approved-answers
Language: en
Content updated: 2026-11-19
Description: Turn scattered business information into an owned, current and testable answer source, with clear limits for live data, conflicts, missing answers and human fallback.

AI voice agents

# How to build an approved knowledge base for an AI voice agent

An AI voice agent should not invent an answer from an old webpage, a sales brochure and a staff note that disagree. Before it handles factual questions, create an approved knowledge base with named owners, source priority, effective dates and a clear response when the information is missing. The goal is not to upload every company document. It is to give callers short, current answers that the business can trace, review and change.

By SVS Digital · Published 17.11.2026

## Start here

Start with the call reasons the agent is allowed to handle. For each factual answer, record the approved wording or source, owner, audience, effective date, review date and fallback. Keep opening hours, service scope and preparation instructions in curated documents; fetch availability, order status, account details and other changing facts from an authorised live system. Resolve conflicting sources before indexing them. Configure the agent to say that it cannot confirm an unsupported answer and route the next step appropriately. Test retrieval with real spoken questions, paraphrases, corrections, stale facts and deliberate conflicts, then repeat the tests whenever a source or agent configuration changes.

## 1. Define which questions the agent may answer

List the most common call reasons and mark each as answer, collect details, perform an approved action or hand over. A knowledge base is suitable for factual questions such as opening hours, service coverage, visit preparation and published policies. Access to a knowledge base should not silently authorise the agent to diagnose a fault, approve an exception, quote an unapproved price or make a commitment that staff would review.

Create an answer register with one row per topic: caller question, approved source, business owner, allowed answer, important qualifier, prohibited claim and fallback. This becomes the operating boundary for content editors, agent builders and testers. A document being technically accessible does not mean every sentence is appropriate to say to a caller.

- Common caller question and permitted outcome.

- Approved source and named business owner.

- Required qualifier, location or customer type.

- Fallback when the question is outside scope.

## 2. Establish one source hierarchy and separate live data

Decide which source wins when the public website, internal procedure and staff memory differ. Prefer a small approved source for each topic rather than several overlapping files. Give documents stable names and identifiers so an update replaces the intended content instead of adding another version. Google documents stable document IDs, source metadata and supporting links; ElevenLabs allows approved documents to be reused across agents.

Separate relatively stable knowledge from live business state. General service descriptions may belong in a reviewed document. Today's available appointment slots, current order status, stock, customer balance or employee availability should come from an authorised system at call time. If that system cannot be checked, the agent should not turn an old document into a live confirmation.

- One authoritative source per topic where possible.

- Stable document identity, visible version and named owner.

- Static information separated from live operational data.

- No live confirmation without a successful system result.

[Google Cloud Dialogflow CX: Data stores](https://docs.cloud.google.com/dialogflow/cx/docs/concept/data-store)[ElevenLabs: Manage knowledge-base documents](https://elevenlabs.io/docs/eleven-agents/customization/knowledge-base/manage-documents)

## Need a reliable answer source for your AI voice agent?

Bring your common call reasons, current FAQs, service information and the systems that hold live data. SVS Digital can help define the answer register, source hierarchy, fallback rules, update workflow and a scoped retrieval test set before a voice-agent pilot.

## 3. Write source content for a spoken answer

A caller cannot scan a long policy page. Put the direct answer first, then the condition that changes it and the next step. Use the same terms callers use for services, locations and dates. Expand abbreviations, write numbers and times unambiguously, and split unrelated topics. Preserve the detailed policy for staff, but give the agent a concise approved explanation rather than asking it to summarise a dense contract during a call.

Add metadata that helps editors and retrieval: topic, service or location, language, audience, effective date, review date, source URL and owner. Google recommends single-language data stores for best performance and supports titles, URLs and metadata. Treat each language as editorial content; test local terminology instead of assuming a literal translation retrieves the same passage.

- Direct answer before background detail.

- Conditions, exclusions and next step stated plainly.

- Dates, times, numbers and names are speakable.

- Language, location, owner and review date recorded.

[Google Cloud Dialogflow CX: Data stores](https://docs.cloud.google.com/dialogflow/cx/docs/concept/data-store)

## 4. Treat retrieval as a lookup, not proof

Retrieval-Augmented Generation selects passages relevant to the caller's question and gives them to the model. ElevenLabs documents indexing, retrieval on every turn or on demand and configurable retrieval limits. That mechanism can make a large source usable, but attaching or indexing a file does not prove that the correct passage will be selected for every accent, paraphrase or multi-part question.

Define the response for no useful source, two conflicting sources and a source that lacks the caller's condition. The agent can ask one clarifying question, state what it can confirm, create a callback request or transfer according to the separate handoff rules. Google data-store tools support explicit fallback behaviour and can expose source snippets and grounding information. Do not convert low confidence into a confident business promise.

- Indexing completed before the content is relied on.

- No-source and conflicting-source fallbacks are written.

- The agent distinguishes confirmed facts from a next-step request.

- Important answers can be traced to the retrieved source.

[ElevenLabs: Retrieval-Augmented Generation](https://elevenlabs.io/docs/eleven-agents/customization/knowledge-base/rag)[Google Cloud Dialogflow CX: Data store tools](https://docs.cloud.google.com/dialogflow/cx/docs/concept/data-store/handler)

## 5. Use an approval and update workflow

Give every source an owner and a review trigger. A new price list, holiday opening time, changed service area or revised policy should create an update task before the old answer expires. Record who approved the change, what topics it affects, when it takes effect and which agents use the document. Remove or archive the superseded source so two versions cannot compete.

ElevenLabs states that editing a document regenerates search chunks and embeddings, while URL documents can be refreshed or automatically synchronised. Even with automatic refresh, the company still needs editorial approval: a website error can otherwise become a phone answer quickly. Publish in a test environment where possible, verify the index or sync status, run the affected tests and only then use the new source for callers.

- Content owner and approver named.

- Effective, expiry and review dates visible.

- Superseded source removed from active retrieval.

- Affected questions retested after reindexing or sync.

[ElevenLabs: Manage knowledge-base documents](https://elevenlabs.io/docs/eleven-agents/customization/knowledge-base/manage-documents)

## 6. Test the answers and maintain a regression set

For each approved topic, test the expected question, natural paraphrases, a vague question, a correction, a different location or service, an out-of-date premise and a two-part request. Inspect what the agent said and which source it retrieved. Google documents traces containing rewritten queries, snippets and supporting documents; use available traces to diagnose retrieval rather than editing answers blindly.

Keep failed and high-impact calls as anonymised test cases where permitted. Re-run them after source, retrieval, prompt, language or model changes. Review a bounded sample of real calls for unsupported claims, stale facts, overlong answers and repeated fallbacks. Access, retention, caller notice, recording and sector duties require appropriate privacy, security or legal review. A maintained knowledge base reduces avoidable errors; it does not guarantee every generated answer.

- Natural spoken variants and corrections included.

- Stale, conflicting and missing information tested.

- Spoken answer compared with the retrieved source.

- Content and data handling reviewed by responsible specialists.

[Google Cloud Dialogflow CX: Data stores](https://docs.cloud.google.com/dialogflow/cx/docs/concept/data-store)[ElevenLabs: Retrieval-Augmented Generation](https://elevenlabs.io/docs/eleven-agents/customization/knowledge-base/rag)

## Need a reliable answer source for your AI voice agent?

Bring your common call reasons, current FAQs, service information and the systems that hold live data. SVS Digital can help define the answer register, source hierarchy, fallback rules, update workflow and a scoped retrieval test set before a voice-agent pilot.

## Sources and further reading

- [ElevenLabs: Manage knowledge-base documents](https://elevenlabs.io/docs/eleven-agents/customization/knowledge-base/manage-documents)

- [ElevenLabs: Retrieval-Augmented Generation](https://elevenlabs.io/docs/eleven-agents/customization/knowledge-base/rag)

- [Google Cloud Dialogflow CX: Data stores](https://docs.cloud.google.com/dialogflow/cx/docs/concept/data-store)

- [Google Cloud Dialogflow CX: Data store tools](https://docs.cloud.google.com/dialogflow/cx/docs/concept/data-store/handler)

## More guides

AI voice agents

## [From an AI voice call to your CRM: design a reliable post-call handoff](https://www.svs-digital.com/guides/ai-voice-agent-post-call-summary-crm-handoff)

Define the post-call record, identifiers, source labels, duplicate protection, routing and reconciliation that turn an AI voice conversation into useful work for your team.

12.11.2026

AI voice agents

## [Appointment booking rules for an AI voice agent](https://www.svs-digital.com/guides/ai-voice-agent-appointment-booking-rules)

Define booking states, services, availability, required details, confirmations, duplicate protection and fallbacks before an AI voice agent writes to your calendar.

07.11.2026

AI voice agents

## [How to test an AI voice agent before it handles real calls](https://www.svs-digital.com/guides/test-ai-voice-agent-before-launch)

Build a pre-launch test matrix for conversation quality, names and numbers, business actions, phone failures and human review before an AI voice agent answers customers.

03.11.2026

---

SVS Digital · Dunajska cesta 5, 1000 Ljubljana, Slovenia
Contact: [uros@svs-digital.com](mailto:uros@svs-digital.com)
[All resources](https://www.svs-digital.com/ai)
