Voice Agent Knowledge Base Hong Kong Design Guide
By Ilya Laptev, Co-Founder, Genium Group · Published
OFCA guidelines on clear information disclosure force Hong Kong businesses to keep every answer traceable. A AI Voice Phone Agents system stands or falls on the voice agent knowledge base that feeds it facts during live calls.
Why Your Voice Agent Knowledge Base Is a Hidden Risk Surface
Most Hong Kong SMEs test the voice first and the knowledge base last. That order creates the failures we see in clinic reception and property agency lines. The model itself rarely hallucinates when the retrieval returns one clean fact. When it returns three overlapping paragraphs or a pricing table with footnotes, the agent invents the missing piece to keep the conversation moving.
Latency compounds the problem. Every extra chunk retrieved adds 200-400 ms before the first word leaves the speaker. In a market where callers expect answers inside eight seconds, that delay turns into dropped calls. Voice agent latency budgets show the KB lookup is often the largest single contributor once the model and TTS are already tuned.
PDPO adds another constraint. If the knowledge base holds client examples or raw notes, every retrieval risks exposing personal data that should never have been indexed. The PCPD’s AI Model Personal Data Protection Framework sets out practical recommendations for organisations that use AI.
The rule for Hong Kong operators is simple. The smaller and more explicit the voice agent knowledge base, the lower the regulatory surface.
What Belongs in a Voice Agent Knowledge Base (And What Does Not)
Start by separating three layers. Global prompts and tools handle tone, escalation paths, and booking actions. The voice agent knowledge base exists only for low-frequency facts that change infrequently and must be quoted verbatim.
For clinics this usually means appointment types, required documents, and referral rules. Law firms need intake criteria and fee-earner availability. Property agencies store unit descriptions and viewing protocols.
Exclude anything that belongs in prompts: escalation phrases, refusal scripts, or brand voice instructions. Exclude draft policies, internal emails, and any document that answers more than one discrete question. Exclude marketing copy that uses ranges or conditional language. These items make retrieval noisy and give the model permission to improvise.
voice agent kb best practices therefore begin with an audit: list every question a caller actually asks in a week, map each to a single source document, and delete the rest. The result is rarely more than thirty short entries for a typical Hong Kong practice.
One Fact Per Document: The Core Design Pattern
Long multi-topic documents are the fastest way to trigger hallucinations. When a caller asks about cancellation fees, the retriever may surface a page that also contains opening hours and parking instructions. The model then has three possible anchors and often blends them. The fix is mechanical: one question, one answer, one file.
Write each entry as a single declarative sentence followed by a short clarifying paragraph. No ranges, no footnotes, no cross-references. The same pattern works for kb for clinic voice agents (“We require a referral letter for specialist appointments”) and law firm call agent kb entries .
Property agency voice kb follows the identical rule. One document per unit type, one document per viewing policy, one document per commission structure. The retrieval engine can then return the exact chunk with high confidence and the agent simply reads it back.
Structuring, Chunking, and Tagging to Prevent Hallucinations
Markdown headings and bullet lists improve chunk boundaries. Use H2 for the core question and a single paragraph or short list for the answer. Add metadata tags: service line, regulatory domain, update date, and refusal instruction. The refusal tag tells the agent what to say when the fact is absent: “I don’t have that detail; let me transfer you.”
Recursive chunking at 200-300 tokens with 20 % overlap keeps each chunk self-contained. Overlap stops the model from losing the question context when the answer sits near a boundary. Anthropic’s write-up on contextual retrieval notes that chunk size, boundaries and overlap all affect retrieval results.
Give every chunk the same metadata tags. Then filters can drop whole categories, such as internal-only notes, before the search starts. OpenAI’s retrieval guide shows how attribute filters narrow results before semantic search.
ai phone agent faq design benefits directly from this discipline. A well-tagged FAQ set becomes the voice agent knowledge base itself. Clinics that maintain separate FAQ and KB repositories usually discover the two diverge within weeks; merging them under one schema removes that drift.
Handling Pricing, Policies, and Compliance in HK / APAC
Pricing pages remain the highest-risk content. The safer pattern is to move variable fees into a tool that returns a calculated quote or to keep only fixed, published rates in the knowledge base.
PDPO compliance requires that any example used in a policy document be anonymised or removed. OFCA expectations around clear information mean the agent must be able to cite the exact source document when challenged. Both obligations are easier to meet when each fact lives in its own file with a clear update date.
Operational maintenance closes the loop. Review the voice agent knowledge base weekly against call transcripts. Any question the agent could not answer becomes a new one-fact document. Any answer that needed a human correction becomes a revised document.
Warm-transfer failure modes drop sharply once the knowledge base stops producing ambiguous retrievals.
Conclusion
A voice agent knowledge base succeeds when it is small by design, strictly one fact per document, and kept up to date against real call data. Hong Kong practices that treat the knowledge base as infrastructure, not marketing copy, see fewer hallucinations and lower latency. They also get clearer audit trails under PDPO and OFCA scrutiny.
Audit your knowledge base against the patterns above and map the gaps to your actual call volume. Start the review at https://genium-group.com/services/voice-agents.
FAQ
How do I structure a voice agent knowledge base so my AI phone agent stops hallucinating pricing?
Remove every pricing statement that contains ranges, conditions, or footnotes. Replace them with either fixed published rates in their own one-fact documents or with a tool that returns a calculated quote. The voice agent knowledge base should contain only numbers the agent can read verbatim without interpretation.
What should I leave out of a voice agent knowledge base for my clinic or law firm?
Leave out internal instructions, draft policies, client examples containing personal data, and any document that answers more than one question. These items increase retrieval noise and create PDPO exposure. Keep only the facts callers actually request and that change infrequently.
Why do long multi-topic documents perform poorly in voice AI retrieval?
A document covering three unrelated topics rarely matches any single caller intent with high confidence. The retriever returns a mixed chunk, and the model blends the sections. Hallucinations follow. Splitting into one-fact documents lets the system surface the precise answer every time.
How often should I review and update my voice agent knowledge base content?
Review weekly against call transcripts and monthly against policy changes. Every unanswered or wrongly answered question becomes a new or revised one-fact document. This cadence keeps the voice agent knowledge base in line with what callers actually need, not with internal assumptions.
Can I safely expose discounts and variable fees in my voice agent knowledge base?
No. Discounts and variables force the model to decide applicability. Move them into a pricing tool or publish only the fixed rate that applies to the majority of callers. The knowledge base remains a source of unambiguous facts the agent can quote directly.
Talk to us
Want to see how this would work in your business? Tell us what you need and we will get back to you within 24 hours.
Our service: AI Receptionist. Answers your main line on the first ring in English, Cantonese or Mandarin, and routes what needs a person.
Voice agents · Voice Agent Latency Budget Hong Kong Calls · AI Warm Transfer Hong Kong Failure Modes · Ai agents · Ai receptionist · Appointment Reminder Calls Hong Kong Benchmark · Number Porting Hong Kong: Twilio vs SIP for Agents · More articles · Talk to our team