How to use Custom Tags to filter Bot Metrics

In this How-To, we will explore a scenario in a Banking Bot, where, various metrics will be used to analyze the bot performance.

For details on what Bot Metrics are and how they are implemented in the Kore.ai Bots platform, refer here.

Problem Statement

A Banking Bot client can track the following metrics:

• Frequent Intents used by customers;
• Usage split based on the customer type;

This document gives a detailed step-by-step approach to viewing the above statistics from the Bot Metrics page and apply filters based on Custom Meta Tags.

Pre-requisites

• Bot building knowledge.
• Custom Meta Tags usage, refer here for more.
• A Banking Bot with the dialogs as mentioned below:
  • Transfer Money – Dialog task walking the user through the steps in transferring money.
    In this diaog, we have included a Script node to add Custom Meta Tag, TransferValue, based upon the amount transferred. Following script was used:

    if(context.entities.TransferAmount[0].amount > 50000){
       tags.addSessionLevelTag("TransferValue","HighValue");
    }
    
    if(context.entities.TransferAmount[0].amount > 10000)
    tags.addSessionLevelTag("TransferValue","MediumValue");
    
    if(context.entities.TransferAmount[0].amount > 0)
    tags.addSessionLevelTag("TransferValue","LowValue");

    

  • Manage Payee – For the user to manage their payee list.

    Here we have a Script assigning customerType meta tags:

    if(context.custType == 3){
       tags.addUserLevelTag("CustomerType","Premium");
    }
    if(context.custType == 2){
       tags.addUserLevelTag("CustomerType","Gold");
    };
    if(context.custType == 1){
       tags.addUserLevelTag("CustomerType","Regular");
    };

Implementation

Let us consider how each of the metrics mentioned in the problem statement can be viewed from the Metrics.

1. From the left navigation panel, under Analyze click Metrics.
2. The page will display the Intent found metrics to include successful utterances, intents, userid of the user,  and other details.
3. These are the successful intents in the past one week as captured by the Bot.
4. Click the Filter button to customize the report.
   
5. Customize the Date period, Languanges, Channel, and the Task/Intent you want to capture.
6. Since we want to track the Transfer Money intent, we have selected accordingly.
   
7. We want the report of all the interactions by the Gold and Premium customers doing High Value money transfers.
8. For this select the Custom Tags as follows (you can select the tag name and tag value from the type ahead drop down:
   • Customer Type – Gold, Premium
   • Transfer Value – HighValue

   

9. On Applying the filter, you will see the updated Metrics page. Note that tags filter is applied only on the published version on the bot.
10. You can repeat the above steps for all metrics like Intent not found, Failed task, etc..