Providing access to a knowledge base through AI

The business software discussed here already had a very extensive knowledge base in the form of a web portal. Searching this documentation was sometimes difficult because users had to use the « right words » to find the most relevant pages. This is where AI comes into play and opens up many new possibilities. The system I implemented is based on a RAG (Retrieval-Augmented Generation) architecture.

In practical terms, I developed two applications:

• A program for preparing the information (vectorisation), written as a .NET console application and run daily on a Linux server.
• A web portal developed in Blazor that provides a conversational interface with the AI.

Information sources

The software’s knowledge base is, of course, the primary source of information. But other content was added to enrich the documentation available to the model: old legacy manuals, selected pages from the commercial website, explanations of the program’s general options available within the user interface and, finally, the SQL database schema (in DDL format).

Preparing the content

To be usable by the AI, all this content must be divided into chunks (chunking). However, the knowledge base also contains C#, SQL and XAML code snippets. These must first be extracted, as splitting such code into chunks would be counterproductive. The content is normalised and all HTML tags are removed. The resulting chunks are stored in a Qdrant vector database using .NET code. The Qdrant server runs on an Ubuntu server.

Conversational portal

This portal is developed in Blazor. Users enter their questions, accept the terms of use and complete a captcha. Blazor is ideal for progressively displaying the AI’s response. This is important because, without progressive output, the reasoning process could appear to take a long time.

Paradoxically, the most difficult part to fine-tune is the system prompt, i.e. the instructions that tell the AI how to respond, what it must not say, how it should structure the content it returns so that URLs to the knowledge base can subsequently be associated with it, …

In practice, a whole series of optimisations are also necessary:

• Detecting whether the question refers to a very specific page.
• Determining whether the question contains variable names, parameters, … that are difficult for semantic search to recognise. In that case, a traditional search (LIKE %) must be added.

The code normalises the question and vectorises it in the same way as during the content preparation process. The search is then performed, first within the textual content and subsequently within the code snippets associated with the pages found during the previous step. Additional weighting is applied if the first result corresponds to a chunk that contains a page title.

All these results form the potentially relevant documentation sent to the AI (ChatGPT). The response is progressively displayed to the user. Once it has been fully generated, the user can continue the conversation and the same process then begins a new iteration.

This portal is capable of composing SQL queries from instructions given in natural language. This is why it has access to the database schema. In practice, it has also proved capable of producing C# code based, among other things, on the examples included in the knowledge base.

A RAG architecture can therefore transform existing documentation into a genuine conversational knowledge base. AI does not replace the information accumulated over the years: it gives it a new dimension by making it possible to query that information in natural language, combine sources of very different kinds and even use that knowledge to produce SQL or code. The quality of the result then depends as much on the model being used as on the care taken in preparing, retrieving and selecting the information provided to it.