RAG knowledge assistant · scoped engineering service

Knowledge assistants that retrieve evidence before they answer

Kubto builds RAG assistants around approved sources, document permissions, hybrid retrieval, source citations, answer-or-decline policy, prompt-injection boundaries, evaluation, and a clear path to human help.

Engagement boundary: This is a custom engineering service rather than a universal chatbot or ticket-deflection guarantee. Sources, connectors, channels, identity, models, hosting, licensing, data use, support, and operational ownership are scoped per engagement.

Plan a RAG readiness review Review the RAG architecture

Product dashboard

RAG knowledge assistant · scoped engineering service

Monitored

Sessions

1.4k

Resolved

72%

Escalated

Assistant quality

Last 30 days

Return policyCited answer96%

Product comparisonCatalog source89%

Order issueHuman handoffQueued

Dashboard metrics are illustrative. Final KPIs, data sources, thresholds, and alerts are defined during discovery.

Who it is for

Teams with a defined operating problem

The best starting point is usually a real workflow, a known constraint, and someone who owns the outcome.

Support and knowledge owners

Teams responsible for approved answers, content quality, escalation, user experience, policy, and the workflow after the assistant cannot answer.

Security, data, and engineering teams

Owners of source access, identity, connectors, retrieval, model policy, deployment, observability, incident response, and production support.

Use cases

Where this capability fits

Each pattern is checked against the data you have, the systems involved, the effort to adopt it, and the risk of getting it wrong.

Customer support assistance

Answer product, delivery, return, troubleshooting, and policy questions from approved content, then escalate with context when the workflow requires a person.

Employee knowledge access

Retrieve permitted policies, procedures, product documentation, project information, and internal guidance according to the user’s source access.

Product and technical assistance

Find specifications, compatibility evidence, manuals, and relevant products while separating sourced facts from generated explanation.

Agent-assist workflows

Give human service or operations teams retrieved evidence, draft responses, case summaries, and next-step suggestions without automatically executing sensitive actions.

Capabilities

What the implementation must account for

The useful shape depends on the source data, user journey, platform limits, controls, and the team that will run it.

ACL-aware ingestion

Preserve stable source identity, document and user permissions, provenance, version, deletion, and freshness through parsing and indexing.

Hybrid evidence retrieval

Combine lexical and vector candidates, metadata and permission filters, fusion, reranking, diversity, and parent-document context based on evaluation.

Cited answer policy

Require the assistant to answer from retrieved evidence, distinguish sources from generated explanation, cite usable locations, and decline when support is insufficient.

Prompt-injection boundaries

Treat documents and user input as untrusted, isolate instructions from evidence, constrain tools, validate output, and test adversarial content.

Human escalation

Route uncertain, sensitive, restricted, high-impact, or user-requested cases with conversation context, sources, reason, and an owned queue.

Quality and content operations

Review retrieval failures, unsupported answers, citation issues, content gaps, source freshness, user feedback, incidents, cost, and release changes.

Business outcomes

Define the baseline before claiming improvement

Strong outcomes need a baseline. Before anyone claims improvement, the team should know what is being measured and under which conditions.

Faster access to approved evidence

Help customers and employees reach relevant source material without requiring them to know which repository or document contains it.

Measure: Task completion, time to evidence, source coverage, successful-answer rate, user feedback, and escalation against the current workflow.

Safer answer behavior

Make decline, citation, permission, restricted-topic, and human-escalation behavior part of the product contract.

Measure: Unsupported-answer rate, citation correctness, ACL leakage tests, restricted-topic handling, escalation quality, and reviewed incidents.

An actionable knowledge backlog

Turn unanswered questions and retrieval failures into evidence for content, metadata, permissions, taxonomy, and process improvement.

Measure: Repeated failure categories, missing-content volume, source-owner response, correction lead time, and regression closure.

Architecture

Reference flow for an ACL-aware RAG assistant

A RAG system is a chain of source, permission, parsing, retrieval, prompt, model, channel, evaluation, and operations decisions—not a single model call.

01
Sources and identity
Connect approved repositories, assign stable identities, synchronize source permissions and versions, and define deletion, retention, and freshness behavior.
02
Parse and index
Extract usable structure, retain headings and provenance, choose chunk or parent-child strategy, create representations, and attach enforceable metadata.
03
Retrieve evidence
Authenticate the user, apply permissions, retrieve complementary candidates, fuse and optionally rerank them, and assemble diverse supported context.
04
Answer under policy
Separate system policy from untrusted content, generate within evidence and channel constraints, validate output, cite sources, or decline and escalate.
05
Evaluate and operate
Record appropriate evidence and traces, review failures, gate releases, monitor sources and dependencies, and assign content and operational corrections.

If source permissions cannot be represented and enforced reliably, the affected content should not be available through the assistant. Post-generation masking is not an adequate access-control design.

Technical design

Decisions documented before production

The exact technologies remain an architectural choice. The engagement documents why each component is selected, how it fails, and who owns it.

Connector and sync contract

Define source scope, API limits, incremental sync, webhooks or polling, stable IDs, versions, deletes, permission changes, failures, reconciliation, and backfills.

Parsing and chunk strategy

Preserve document structure, tables, headings, pages, parent-child context, citations, metadata, language, and OCR quality where relevant.

Hybrid retrieval pipeline

Evaluate query transformation, lexical/vector candidates, reciprocal-rank or other fusion, metadata filters, reranking, diversity, context assembly, and no-result handling.

Answer and citation contract

Specify supported question types, source requirements, citation format, quote limits, uncertainty language, decline, restricted topics, and escalation payload.

Injection and tool controls

Separate instructions from retrieved text, sanitize rendering, constrain tools and URLs, validate structured output, test indirect injection, and require approval for actions.

Evaluation and operations

Version golden and adversarial cases, capture retrieval and answer traces appropriately, monitor source freshness, quality, cost, dependencies, incidents, and content gaps.

Integration surface

Fit the system to the existing estate

Named technologies indicate common integration points, not a universal compatibility guarantee. Versions, APIs, limits, and connector scope are verified during discovery.

Knowledge sources

Document systems, help centers, wikis, file stores, databases, product catalogs, policies, tickets, and APIs after access and connector review.

Identity and permissions

SSO, directory groups, application roles, repository ACLs, tenant context, and service identities that can be enforced end to end.

User channels

Websites, portals, helpdesks, collaboration tools, internal applications, and agent desktops through supported APIs and approved channel policies.

AI and operations

Embedding and generation models, lexical/vector retrieval, queues, caches, observability, evaluation, ticketing, and incident systems.

Deployment and ownership

Operate sources, retrieval, models, and channels as one service

A usable answer depends on every stage. Ownership and diagnostics must cross repository, identity, index, model, application, and support boundaries.

Environment, network, identity, secrets, provider, region, data-residency, and retention decisions
Connector, ACL sync, parsing, index, model, prompt, channel, and evaluation versioning
Dependency failure, stale source, no-evidence, fallback, escalation, rollback, and disaster recovery
Content owner, system owner, support, security, cost, incident, and improvement responsibilities

Security and boundaries

Retrieved content is evidence and also an attack surface

The assistant must preserve authorization while treating source and user content as potentially incorrect, malicious, stale, or policy-restricted.

Enforce ACL and tenant filters before evidence reaches the model
Separate system instructions from documents and test direct and indirect prompt injection
Minimize personal and restricted data; document provider use, retention, deletion, and audit
Require decline or human escalation when evidence, authorization, or policy support is insufficient

Delivery

A scoped path from evidence to operation

Each phase produces reviewable artifacts. Timing and team composition depend on data access, platform complexity, risk, and procurement requirements.

RAG readiness assessment

Inventory questions, users, sources, permissions, content quality, current support workflow, channels, risk, and baseline.

Deliverables: Question and source map, ACL findings, content-readiness report, risk classification, baseline, and prioritized scope.

Retrieval and answer design

Define connectors, identity, parsing, indexes, ACL enforcement, retrieval, citations, answer policy, escalation, evaluation, and operations.

Deliverables: Reference architecture, threat model, source and answer contracts, evaluation plan, operating model, and scope.

Evidence-based pilot

Implement a bounded source and user group with production-like permissions, adversarial content, representative questions, traces, and review.

Deliverables: Working pilot, retrieval and answer results, security findings, user feedback, cost profile, and production decision.

Production and knowledge operations

Harden sync, access, deployment, channels, monitoring, incidents, content corrections, release gates, training, and ownership.

Deliverables: Production assistant, evaluation suite, dashboards, runbooks, content workflow, training, and ownership matrix.

Evaluation methodology

Test quality, risk, and operations together

A production decision should combine offline quality checks, workflow acceptance, security review, operational testing, and business measurement.

Retrieval quality

Measure evidence recall, ranking, diversity, metadata and ACL correctness, missing support, stale sources, and retrieval failure categories.

Answer groundedness

Review factual support, citation correctness, completeness, attribution, unsupported statements, decline behavior, and user-task fit.

Security and policy

Test ACL leakage, direct and indirect injection, restricted topics, unsafe links or tools, personal data, refusal, escalation, and audit evidence.

Operational readiness

Exercise source changes, permission changes, deletes, dependency failure, model changes, rollback, monitoring, incidents, cost, support, and content ownership.

Questions

What buyers usually need to confirm

How do you prevent the assistant from seeing restricted documents?

Source permissions are synchronized into stable metadata and enforced before evidence reaches the model. The design includes permission-change and leakage tests and fails closed when authorization cannot be established.

Do citations eliminate hallucinations?

No. Citations make claims inspectable, but the system still requires retrieval evaluation, groundedness review, answer policy, adversarial tests, and decline behavior. Citation presence and citation correctness are measured separately.

Can this automatically deflect a fixed share of tickets?

No fixed outcome is responsible without the client’s question mix, content quality, channel behavior, and baseline. Kubto measures successful task completion, escalation, user feedback, safety, and support impact in the actual workflow.

Continue evaluating

Start with questions, sources, and permissions

Share representative questions, repositories, user groups, restricted topics, escalation workflow, channels, and content owners. Kubto will scope a readiness assessment before a build.

Plan a RAG readiness review