1. Home
  2. Kore.ai Conversational Platform
  3. Bot Builder Tool
  4. Analyzing Your Bot

Analyzing Your Bot

The Analyze page helps you gain in-depth insights into your bot’s performance in identifying and executing tasks. It lets you view necessary information for user utterances that matched and didn’t match with intents. Also, it shows the tasks that failed to execute after matching an intent as well as the backend performance of the script and service nodes. The information is available on separate tabs from where you can obtain user details, conversation details, NLP analysis, and the chat history of the user session.

To open the Analyze page, hover over the side navigation panel and click Analyze.

Filter Criteria

You can filter the information on the Analyze page using the following criteria.

Note: The filter criteria differs slightly between different tabs.
Criteria Description
User ID The UserID of the end user related to the conversation.
Date period The page shows the conversations from the last 7 days by default. To filter the conversation to just the ones from the last 24 hours, click Today. To switch back to the sessions from the last 7 days, click Last 7 days.
Language If it is a multi-language bot, you can select specific languages to filter the conversation that occurred in those languages. The page shows the conversations that occurred in all enabled languages by default.
Channel Select specific channels to filter the conversation that occurred in those channels. The page shows the conversations that occurred in all enabled channels by default.
Task/Intent Select specific tasks or intents to filter the conversation related to those tasks or intents. The page shows the conversations related to all tasks or intents by default.
Utterance Type Select the Trained option to filter the conversations that only contained trained utterances to the bot.
To view the conversations that involved untrained utterances, click Not Trained. The page shows the conversations related to both by default.
Developer Interactions Select Include Developer Interactions if you want to include developer interactions in the results. By default, the developer interactions aren’t included. Developers include both the bot owner and shared developers.

Identified and Unidentified Intents

The primary details, filter criteria, and the advanced details for both the Intent Found and Intent Not Found are similar, with minor differences. You can also train the bot for any utterances directly from these tabs.

Primary Details

Field Description
Utterances The actual utterance entered by the user. The details in the tab are grouped by utterances by default. To turn off grouping by utterance, click the Utterances header and turn off the Group by Utterances option.
Intent

(applies only to the Intent Found tab)

The intent that was identified for the user utterance. You can take a look at the identified intent and the user utterance to determine if they are a right match. If not, you can train the bot from here. To turn on grouping by intent, click the Intent header and turn on the Group by Task option.
UserID The UserID of the end user related to the conversation.
Language The language in which the conversation occurred.
Date & Time The date and time of the chat.

Training the Bot

You can train an intent from both the Intent Found and Intent Not Found tabs. To do so, hover over a row in any of these tabs, and click the Train icon. It opens the Test & Train page from where you can train the bot. For more information, read Testing and Training a Bot.

Failed Tasks

The Failed Tasks tab shows the following details related to the task that was identified but failed to execute for any reason:

Field Description
Utterances The actual utterance entered by the user. The details in the tab are grouped by utterances by default. To turn off grouping by utterance, click the Utterances header and turn off the Group by Utterances option.
Task Name The task that was identified for the user utterance. To turn on grouping by task name, click the Task Name header and turn on the Group by Task option.
Failure Point Nodes or points in the task execution journey where the failure occurred resulting in the task cancellation or user drop. Click an entry to view the complete conversation for that session with markers to identify the intent detection utterance and the failure/drop-out point. Depending on the task type, click Failure Point shows more details.
Type of Issue Shows one of these options as the reason for failure:

  • Task aborted by user
  • Alternate task initiated
  • Chat Interface refreshed
  • Human agent transfer
  • Authorization attempt failure – Max attempts reached
  • Incorrect entity failure – Max attempts reached
  • Script failure
  • Service failure
User ID The UserID of the end user related to the conversation.
Language The language in which the conversation occurred.
Date & Time The date and time of the chat.

Advanced View – NLP Analysis and Chat History

For all the user utterances listed under the Intent found, Intent not found, and Failed Task tabs, you can open advanced details related to the user session with the following sub-tabs:

  • Details: Shows the basic details of the session along with a JSON file that includes the NLP analysis for the conversation.
  • NLP Analysis: Provides a visual representation of the NLP analysis including with intent scoring and selection. For more information, read Testing and Training a Bot.
  • Chat history: Directs you to the exact message or conversation for which the record is logged and shows the entire chat history of the user session.
NLP Analysis and Chat History will be available only for the chat logs created after v6.1.0

Performance

The Performance tab shows the following information related to the backend performance of the bot:

Field Description
Script/ Service Name The name of the service or script within the task that got executed in response to the user utterance. To turn on grouping by components to which these scripts or services belong, click the Script/ Service Name header and turn on the Group by Component option.
Type Shows whether it is a script or service.
Task The task that was identified for the user utterance. To turn on grouping by task name, click the Task Name header and turn on the Group by Task option.
Total Runs The total number of times within the date period that the script or service was run for any user utterances.
Average Response Time The average response time of the script or service in the total number of runs.
Success Ratio The percentage of the service or script runs that got executed successfully.

Advanced Performance Details

Clicking a service or script name opens advanced details dialog for the service which lists each instance of its run along with separate tabs for successful and failed runs. Analyzing the average response time of different runs gives you insights into any aberrations in the service or script execution. Click any row to open the JSON response associated with the service or script run.

Exporting the Data

You can export the data present in Bot Analyze page to a CSV file by clicking the Export icon on the top right corner of the page.

Once you click the icon, the export process starts and you can use the Status Tracker dock to track the export progress. After completion of the export, the dock shows export status and if it’s successful provides a link to download the file.

The download includes the information present on the selected tab as well as the detailed analysis based on the selected filters.

Was this article helpful to you? Yes 3 No 3