{"id":60598,"date":"2023-12-15T13:27:16","date_gmt":"2023-12-15T13:27:16","guid":{"rendered":"https:\/\/developer.kore.ai\/v10-1\/?p=60598"},"modified":"2024-02-22T09:23:47","modified_gmt":"2024-02-22T09:23:47","slug":"llm-integration","status":"publish","type":"post","link":"https:\/\/developer.kore.ai\/v10-1\/docs\/bots\/nlp\/llm-integration\/","title":{"rendered":"LLM Integration"},"content":{"rendered":"
<\/div>\r\n\t
\r\n\t\t
\r\n\t\t\t

To use LLM and Generative AI features, you must configure the integration with a pre-built or custom LLM.<\/span><\/p>\n

Pre-built LLM Integration<\/span><\/h2>\n

The XO Platform offers seamless <\/span>integration<\/span> with leading AI services like Azure OpenAI, OpenAI, and Anthropic. Utilizing pre-configured prompts and APIs, you can effortlessly tap into the core capabilities of these services. Although you can customize certain elements, the overall structure adheres to a standardized format for simplicity. You can quickly realize the potential of LLM with the XO platform’s plug-and-play integration with premium AI services.<\/span><\/p>\n

Configure Pre-built LLM Integration<\/span><\/span><\/h3>\n

Steps to configure a pre-built LLM:<\/span><\/p>\n

    \n
  1. Go to <\/span>Build <\/b>> Natural Language <\/b>> Generative AI & LLM<\/b> > LLM Integrations<\/strong>.
    \n\"\"<\/a>
    \n<\/span><\/span><\/li>\n
  2. Choose the LLM you want to configure<\/span>.<\/span>\n
      \n
    1. Steps to configure Azure OpenAI:<\/span>\n
        \n
      1. Click <\/span>Configure Now<\/b> for <\/span>Azure OpenAI.
        \n        \"\"<\/a><\/span>
        \n<\/span><\/li>\n
      2. On the Authorization<\/strong> tab, enter the details like API Key<\/strong> and the User Sub Doman<\/strong>. Toggle on the required\u00a0model<\/strong> and enter Deployment ID<\/strong>.<\/span><\/li>\n
      3. Read the <\/span>Policy Guidelines<\/b>, select the checkbox, and click <\/span>Save<\/b>.<\/span><\/li>\n<\/ol>\n<\/li>\n
      4. Steps to configure OpenAI:<\/span>\n
          \n
        1. Click <\/span>Configure Now<\/b> for <\/span>OpenAI.
          \n          \"\"<\/a><\/span>
          \n<\/span><\/li>\n
        2. On the Authorization<\/strong> tab, enter the API <\/strong><\/span>Key<\/b>.<\/span><\/li>\n
        3. Read the <\/span>Policy Guidelines<\/b>, select the checkbox, and click <\/span>Save<\/b>.<\/span><\/li>\n<\/ol>\n<\/li>\n
        4. Steps to configure Anthropic:<\/span>\n
            \n
          1. Click <\/span>Configure Now<\/b> for <\/span>Anthropic.
            \n            \"\"<\/a><\/span>
            \n<\/span><\/li>\n
          2. On the Authorization<\/strong> tab, enter the API <\/strong><\/span>Key<\/b>.<\/span><\/li>\n
          3. Read the <\/span>Policy Guidelines<\/b>, select the checkbox, and click <\/span>Save<\/b>.<\/span><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<\/li>\n
          4. The configured model is listed in the LLM Integrations section. The status changed to X\/Y<\/em> models configured, where X<\/em> is the number of the model(s) configured, and Y<\/em> is the number of model(s) supported.
            \n<\/span>            \"\"<\/a><\/span><\/li>\n
          5. If you want to configure another model, <\/span>click <\/span>+New Integration<\/b> and select the model.
            \n<\/span>            \"\"<\/a><\/span><\/li>\n
          6. The next step is to e<\/span>nable Co-Pilot<\/a> and Dynamic Conversations<\/a> features.<\/span><\/li>\n<\/ol>\n

            Custom LLM Integration Framework<\/span><\/h2>\n

            The XO Platform now enables enterprises to power up their virtual assistants with any Large Language Model (LLM) of their preference. The bring-your-own (BYO) model framework supports integrations with externally hosted models by third parties as well as models hosted by the enterprises themselves. It allows the creation of custom prompts that are optimized for their specific purposes and models. This generic framework works seamlessly with the Auth Profiles module of the platform, enabling enterprises to use the authentication mechanism of their choice.<\/span><\/p>\n

            Note<\/strong>: Generative AI features are available for English and non-English NLU and VA languages on the Kore.ai XO Platform. However, custom LLM-specific features are currently limited to English.<\/div>\n

            Configure Custom LLM Integration<\/span><\/span><\/h3>\n

            You can integrate a custom LLM and build your Prompts and Requests Library for specific features.<\/span><\/p>\n

            Steps to integrate a Custom LLM:<\/span><\/p>\n

              \n
            1. Go to <\/span>Build<\/b> > Natural Language<\/b> > Generative AI & LLM<\/b> > LLM Integrations<\/b>.<\/li>\n
            2. Click <\/span>Custom LLM<\/b>.
              \n              \"\"<\/a><\/span>
              \n<\/b>If you have already configured a model, click <\/span>+New Integration <\/b>> Custom Integration<\/b>.
              \n              \"\"<\/a><\/span>
              \n<\/b><\/li>\n
            3. On the Configuration<\/b> tab, enter the details like Integration Name<\/b>, Model Name<\/b>, Endpoint<\/b>, and Headers<\/b>.
              \n<\/span>
              \n              \"\"<\/a><\/span><\/li>\n
            4. On the <\/span>Auth <\/b>tab, select an existing authorization profile from the drop-down or create a new profile\u00a0 to use for this request. For more information, see the <\/span>Bot Authorization Overview<\/span><\/a> article.<\/span>
              \n
              \n              \"\"<\/a><\/b><\/b><\/p>\n
              Note<\/strong>: OAuth v2.0 and Kerberos SPNEGO auth profiles are supported for the Custom LLM integration.<\/div>\n<\/li>\n
            5. Click <\/span>Test<\/b>. The Request Body to test LLM Connection\u00a0pop-up is displayed.
              \n              \"\"<\/a><\/span>
              \n<\/span><\/li>\n
            6. Enter the test payload and then click <\/span>Test<\/b> to check the connection. If the LLM connection is successful during the test call, it displays a confirmation message. If not, it shows an error message.
              \n              \"\"<\/a><\/span>
              \n<\/span><\/li>\n
            7. Please <\/span>Read Policy Guidelines<\/b>, select the check box, and then click <\/span>Save<\/b>.<\/span><\/li>\n
            8. The success confirmation message is displayed on the screen. The <\/span>configured model is listed in the LLM Integration section. The status changed to X\/Y<\/em> models configured, where X<\/em> is the number of the model(s) configured, and Y<\/em> is the number of model(s) supported.
              \n              \"\"<\/a><\/span>
              \n<\/span><\/li>\n
            9. The next step is to add Prompts<\/a>.<\/span><\/li>\n<\/ol>\n

              Kore.ai XO GPT Integration<\/span><\/span><\/h2>\n

              The new Kore.ai XO GPT Models module provides fine-tuned large language models optimized for enterprise conversational AI applications. These models have been evaluated and fine-tuned to be accurate, safe, and efficient for production deployment. Initial capabilities include Conversation Summarization and User Query Rephrasing.<\/span>\u00a0Learn more<\/a>.<\/span><\/p>\n

              Note<\/strong>: The Kore.ai XO GPT Module is currently available only for English language interactions. The respective models are available only in our global deployment (US Cloud \u2013\u00a0https:\/\/bots.kore.ai).<\/div>\n

              Enable Kore.ai XO GPT Integration<\/span><\/span><\/h3>\n

              Steps to enable Kore.ai XO GPT Models.<\/span><\/p>\n

                \n
              1. Go to <\/span>Build <\/b>><\/span> Natural Language <\/b>><\/span> Generative AI & LLM<\/b> > <\/span>LLM Integrations<\/b>.<\/span><\/li>\n
              2. Click <\/span>Enable Now<\/b> for Kore.ai XO GPT.
                \n                \"\"<\/a>
                \n<\/span>If you have already configured a model, click +New Integration<\/b> > Kore.ai XO GPT<\/b>.
                \n                \"\"<\/a><\/li>\n
              3. On the <\/span>Models <\/b>tab, toggle on the models as required.
                \n                \"\"<\/a>
                \n<\/span><\/li>\n
              4. Read the Policy Guidelines<\/b>, select the checkbox, and click <\/span>Save<\/b>.<\/span><\/li>\n
              5. The success confirmation message is displayed on the screen. The configured model is listed in the LLM Integration section. The status changed to <\/span>X\/Y<\/span><\/i> models configured, where <\/span>X<\/span><\/i> is the number of the model(s) configured, and <\/span>Y<\/span><\/i> is the number of model(s) supported.
                \n                \"\"<\/a>
                \n<\/span><\/li>\n
              6. The next step is to e<\/span>nable Co-Pilot<\/a> and Dynamic Conversations<\/a> features.
                \n<\/span><\/li>\n<\/ol>\n

                Reset\/Delete the Configured LLM Integration<\/span><\/span><\/h2>\n

                If not using a configured LLM, you can reset Kore.ai XO GPT and prebuilt integration using the <\/span>Reset Configuration<\/b> option. However, you can <\/span>Delete<\/b> custom integrations.<\/span><\/p>\n

                When you reset\/delete the integration, the system does the following:<\/span><\/p>\n