Connect a Knowledge Base to the Chatbot¶

In the previous tutorial, you learned how to build a knowledge base. In this tutorial, you’ll learn how to integrate it with a Large Language Model (LLM) for Retrieval-Augmented Generation (RAG).

What you'll build¶

PineValley Bank needs to develop a chatbot powered by Retrieval-Augmented Generation (RAG) to assist customers in understanding the bank's offerings and products. This chatbot leverages a knowledge base built from the bank's documents, stored in a vector database, and integrates with a Large Language Model (LLM). By combining these technologies, the chatbot can retrieve relevant information and generate accurate, context-aware responses to customer queries, providing instant and personalized assistance.

Now, it's time to design the RAG flow. Follow the steps below to create the RAG integration.

What you'll learn¶

How to create Retrieval-Augmented Generation (RAG) integration API.

Prerequisites¶

You need Visual Studio Code (VS Code) with the Micro Integrator for VS Code extension installed.

Info

See the Install Micro Integrator for VS Code documentation to learn how to install Micro Integrator for VS Code.
You must have completed the Build a Knowledge Base tutorial under Build your first AI Integration before proceeding. Start the Build a Knowledge Base tutorial if you haven’t completed it yet.

Follow the instructions below to modify the existing project to add the RAG functionality.

Step 1 - Create a new API¶

To develop the above scenario, let's get started with creating a new API in the existing BankIntegration project.

Click on the add new API (+) icon in the APIs in the Micro Integrator Project Explorer to create a new API.
Create a new API named RAGChatAPI by giving the name in the Create API form. Click Create to create the API.
In the Service Designer pane, modify the Resource Path to \chat and the Method to POST.

Step 2 - Design the integration¶

Open the Resource View of the newly created API resource by clicking the POST /chat resource under Available resources in the Service Designer.
After opening the Resource View, click on the Start node on the canvas to set an input payload for the integration flow.

Note

Setting an input payload for the integration flow is not mandatory. However, it is recommended, as it will be used to enable expression suggestions, which you will explore in later steps of this tutorial.

Click Add Request, provide the following JSON payload, then click Add. Finally, click Save to complete the input payload setup.
```
{
    "userID": "C567",
    "query": "I want to invest in PineValley bank product"
}
```
Click on the + icon on the canvas to open the Mediator Palette.
Select RAG Chat operation from the AI category in the Mediator Palette.
Fill the form with the following details and click Add to add the RAG Chat operation to the integration flow.

We are going to reuse the same connections that we created in the previous tutorial.

Embedding Model Connection: OPENAI_CONN
Vector Store Connection: KB_CONN
LLM Connection: OPENAI_CONN
Memory Connection: FILE_MEMORY_CONN
User ID: payload.userID (with EX enabled)
User Query/Prompt: ${payload.query}
Overwrite Message Body: true

Now let's add the respond mediator to respond to the client.
Click on the + icon located just after the RAG Chat operation to open the Mediator Palette to add a Respond mediator.

You have successfully updated the integration flow to implement Retrieval-Augmented Generation (RAG) functionality.
For reference, you can review the configured API.

RAG Chat API

Info

You can view the source view by clicking on the Show Source (</>) icon located in the top right corner of the VS Code.

Design ViewSource View

<?xml version="1.0" encoding="UTF-8"?>
<api context="/ragchatapi" name="RAGChatAPI" xmlns="http://ws.apache.org/ns/synapse">
    <resource methods="POST" uri-template="/chat">
        <inSequence>
            <ai.ragChat>
                <connections>
                    <llmConfigKey>OPENAI_CONN</llmConfigKey>
                    <memoryConfigKey>FILE_MEMORY_CONN</memoryConfigKey>
                    <embeddingConfigKey>OPENAI_CONN</embeddingConfigKey>
                    <vectorStoreConfigKey>KB_CONN</vectorStoreConfigKey>
                </connections>
                <userID>{${payload.userID}}</userID>
                <prompt>${payload.query}</prompt>
                <outputType>string</outputType>
                <responseVariable>ai_ragChat_1</responseVariable>
                <overwriteBody>true</overwriteBody>
                <embeddingModel>text-embedding-3-small</embeddingModel>
                <maxResults>5</maxResults>
                <minScore>0.75</minScore>
                <modelName>gpt-4o</modelName>
                <temperature>0.7</temperature>
                <maxTokens>4069</maxTokens>
                <topP>1</topP>
                <frequencyPenalty>0</frequencyPenalty>
                <maxHistory>10</maxHistory>
            </ai.ragChat>
            <respond/>
        </inSequence>
        <faultSequence>
        </faultSequence>
    </resource>
</api>

Step 3 - Run the integration¶

Now that you have updated the integration, it's time to deploy the integration to the Micro Integrator server runtime.

Click the Build and Run icon located in the top right corner of VS Code.

Step 4 - Test the integration service¶

Let us test a scenario where a customer wants to know about the interest rate, minimum balance, and withdrawal rules for a High-Interest Savings Account (HISA) at PineValley Bank.

Once the Runtime Services interface is open, select the RAGChatAPI, and click the Try It button.
Select the /chat resource and click Try it Out to test the API.

Provide a JSON payload and click the Execute button to invoke the API. You may use the following sample payload to test the API.
```
{
    "userID": "001",
    "query": "Can you tell me the interest rate and minimum balance for a High Interest Savings Account?"
}
```

Review the following response to identify the sources retrieved from the vector database. The AI-generated response is available in the content section, while the relevant sources are listed in the sources section of the response.

{
"content": "The High-Interest Savings Account (HISA) at PineValley Bank offers an interest rate of 3.5% per annum and requires a minimum balance of $1,000.",
"tokenUsage": {
    "inputTokensDetails": {
    "cachedTokens": 0
    },
    "outputTokensDetails": {
    "reasoningTokens": 0
    },
    "inputTokenCount": 1073,
    "outputTokenCount": 38,
    "totalTokenCount": 1111
},
"sources": [
    {
    "textSegment": {
        "text": "Article: Overview of High-Interest Savings Account (HISA)\nDocument Title: PineValley Bank – HISA Product Brochure\nLast Updated: March 2025\nDocument Type: Product Brochure\nApplies To: All US residents aged 18 and older\n\nProduct Summary\nA High-Interest Savings Account (HISA) offers a competitive interest rate to help you grow your savings faster while maintaining easy access to your funds.\n\nKey Features\n\n- Interest Rate: 3.5% per annum\n- Minimum Balance: $1,000\n- Withdrawals: Unlimited and free\n- Eligibility: US residents, 18+ years old\n\nOnboarding Requirements\nTo open a HISA with PineValley Bank:\n\n1. Provide a valid government-issued ID\n2. Upload proof of address (e.g., utility bill)\n3. Agree to HISA terms & conditions via e-signature\n\nEstimated completion time: 10 minutes",
        "metadata": {
        "index": "0"
        }
    },
    "metadata": {}
    }
],
"finishReason": "STOP",
"toolExecutions": []
}

Congratulations!
You have now learned how to create a Retrieval-Augmented Generation (RAG) application by integrating a knowledge base with a Large Language Model (LLM) to enhance your integration flow.

What's Next?¶

So far, you have learned to create a RAG application by integrating the knowledge base with the LLM. In the next tutorial, you will learn how to create an AI assistant that can automate tasks, further enhancing your integration capabilities.

Click on the Next button below to continue to the next tutorial.

Next →