Your company chatbot knows everything. Including what it should not.

A useful internal AI knowledge assistant should answer from approved information, show where the answer comes from and respect the access rights of the person asking. Retrieval-augmented generation, usually shortened to RAG, can help connect an answer to company material. It does not automatically make that material current, correct or appropriate for every employee.

The starting question is not how many documents you can upload. It is which question becomes easier for which person, using which sources. An assistant for a new sales colleague and an assistant for the finance team may share an interface while requiring very different information boundaries.

The smallest useful knowledge base

Choose a repeated question family before choosing an entire company archive. For example, a fictional customer-support team might need approved product guidance and escalation rules. Start there. Mixing draft strategy, old price lists, HR records and half-finished meeting notes into the same pilot creates several problems while proving very little about usefulness.

Microsoft’s RAG overview explains how retrieval can ground responses in organisational content and identifies access control among the implementation challenges. The business consequence is straightforward: the source collection is part of the product, not a pile of files someone tidies after launch.

A permission is not a polite instruction

Suppose one employee can read a policy but cannot read a confidential investigation. The assistant should not retrieve the investigation for that employee and then hope a prompt persuades the model to stay discreet. Access needs to be enforced by the surrounding system. The same question from two people can legitimately have different available evidence.

OWASP describes unauthorised retrieval and cross-context leakage among RAG risks, and recommends fine-grained, permission-aware controls. My practical addition is to make access boundaries part of the demonstration: show a permitted answer, a restricted request and an access change, rather than only the happiest possible question.

The document-owner problem

An assistant can deliver a beautifully phrased answer from a document nobody has maintained since the last reorganisation. A citation is helpful because a reader can inspect it. A citation does not make an obsolete document true.

For each source collection, name the owner, the intended readers, the update route and the rule for withdrawal. Ask what happens when a source is replaced or access is revoked. “We will refresh it regularly” needs to become an operational decision someone can carry out.

A pilot test your team can actually run

Here is an original acceptance worksheet for a fictional internal assistant. Replace the examples with approved, non-sensitive material from your own process. It is a starting discussion tool, not a complete security assessment.

Keep the question, expected evidence, permitted audience and reviewer decision with each test. Otherwise the team ends up debating whether the response “sounds good.” A product needs a more reliable standard than the most confident person in the meeting.

Measure the work after the answer

Ask whether the user found the right information and completed the next task. Record how often the answer required correction, how long verification took and where the assistant could not help. A high number of questions is usage; it is not automatically successful knowledge work.

Watch for a less obvious failure: employees stop checking the cited source because the assistant usually sounds sensible. Design the response around a useful decision, with visible limits where needed. A short answer with the right link can be more helpful than a confident paragraph that hides uncertainty.

Give people a route back to a person

If the answer affects pay, contractual commitments, customer promises or other consequential work, decide how it is verified before someone acts. This article does not provide legal advice or certify a deployment. Your own security, privacy and policy owners need to review the relevant parts of the design.

For ordinary unanswered questions, an escalation should carry the question and relevant context to the right owner. It should not create a second inbox that nobody reads. If users must explain the whole problem again, the assistant has moved the work rather than removed it.

Start with one audience and one useful outcome

My background includes enterprise knowledge systems, AI automation and product delivery. For a knowledge-assistant project, I would first clarify the audience, sources, permissions, evaluation and operating owner. You can read how I approach AI integration or explore training that helps teams use a new capability.

The point is not to build a company oracle. It is to help people get a dependable next step without creating a new information problem. The assistant can be friendly. Its access control should be considerably less conversational.

Questions people ask

What is an internal AI knowledge assistant?

It helps employees find and use approved company information, often by retrieving relevant material and generating an answer with sources. The right design depends on the audience, source quality, permissions and task.

Does RAG prevent an AI assistant from making mistakes?

No. Retrieval can provide relevant evidence, but it does not guarantee a correct answer. Test relevance, source freshness, uncertainty handling and the work users do after reading the response.

How should a company chatbot handle confidential documents?

The surrounding system should enforce the user’s access rights during retrieval and throughout the workflow. Do not rely on a conversational instruction as the access boundary. Involve the appropriate security and privacy owners before deployment.