Analyzing Your Bot

The Kore.ai XO Platform provides various dashboards to analyze the real-time data generated by the virtual assistant through its interaction. This data can be useful and provide value for your business.

These dashboards are available within the Analyze section of the Platform and are as follows:

Overview Dashboard: The page summarizes the Conversations Dashboard, Users Dashboard, and Performance Dashboard.
Conversations Dashboard: This page provides insights into the usage and containment of conversation sessions. It focuses on whether requests were successfully answered by the virtual assistant (VA), landed with agents, or were incomplete. You can view many other metrics like the conversation session trend, conversation path analysis, session distribution by channel, VA’s engagement, etc., with the breakup of self-service, drop-off, and agent transfer sessions.
Users Dashboard: The page is a central place for information on new and returning user trends. It also provides insights like total unique users count, returning users count, new users count, and weekly or daily user retention cohort.
Performance Dashboard: This page helps you understand the performance of a virtual assistant. A VA designer needs insights into how many intents were identified, how many tasks were successfully completed using the virtual assistant, how many services or scripts failed during the interactions etc. The performance dashboard provides this essential information.
Custom Dashboard: This page allows you to design your own reports and dashboards.
NLP Insights: This page helps you achieve the following:
- Gain in-depth insights into your VA’s performance in identifying and executing tasks;
- Get a summarized view of VA conversations, intents detected, and drop-offs with the complete VA-user conversation presented as a flow diagram.
Conversations History: This page helps you analyze the user interactions by summarizing the key events identified during the conversation. Intent identifications, entity or confirmation retries, task execution failures, etc., are some key events tracked.
Conversation Flows: This page can be used to visualize the journey of various VA-user interactions across conversation sessions in a given time period. This helps developers understand their assistant’s most frequently used tasks, how users invoke them, popular task execution paths, dropout points, etc.
Conversation Insights: The Conversation Insights dashboard provides a visual user utterances map to easily discover false positives and negatives and identify opportunities to train new intents. The dashboard aims to provide a structured approach for analysts to regularly review and improve the NLU performance. Utterance clusters are formed using deep neural models, and each cluster is assigned a name representative of the underlying utterances. You can also switch to the tabular view to see the list of all intents and the corresponding clusters under these intents. You can categorize the utterances as True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN) and train them for existing or new intents.
Feedback Dashboard: The Feedback Dashboard uses the Feedback Survey feature to help you capture user feedback about your products, services, and overall experience with the virtual assistant. The in-built analytics help you get valuable insights using NPS, CSAT, or any other metrics of your choice. The Feedback Dashboard provides survey results and insights. You can see the distribution trend of feedback scores and the overall score for a given period. You can also review the full feedback results with an option to export them.
Usage Metrics: This page provides the VA activity and bot user overview. It includes both interactive and non-interactive sessions.
Containment Metrics: This page provides insights on whether queries are successfully answered by the Virtual assistant or handled by agents.

Analyzing Your Bot

These dashboards are available within the Analyze section of the Platform and are as follows:

Overview Dashboard: The page summarizes the Conversations Dashboard, Users Dashboard, and Performance Dashboard.
Conversations Dashboard: This page provides insights into the usage and containment of conversation sessions. It focuses on whether requests were successfully answered by the virtual assistant (VA), landed with agents, or were incomplete. You can view many other metrics like the conversation session trend, conversation path analysis, session distribution by channel, VA’s engagement, etc., with the breakup of self-service, drop-off, and agent transfer sessions.
Users Dashboard: The page is a central place for information on new and returning user trends. It also provides insights like total unique users count, returning users count, new users count, and weekly or daily user retention cohort.
Performance Dashboard: This page helps you understand the performance of a virtual assistant. A VA designer needs insights into how many intents were identified, how many tasks were successfully completed using the virtual assistant, how many services or scripts failed during the interactions etc. The performance dashboard provides this essential information.
Custom Dashboard: This page allows you to design your own reports and dashboards.
NLP Insights: This page helps you achieve the following:
- Gain in-depth insights into your VA’s performance in identifying and executing tasks;
- Get a summarized view of VA conversations, intents detected, and drop-offs with the complete VA-user conversation presented as a flow diagram.
Conversations History: This page helps you analyze the user interactions by summarizing the key events identified during the conversation. Intent identifications, entity or confirmation retries, task execution failures, etc., are some key events tracked.
Conversation Flows: This page can be used to visualize the journey of various VA-user interactions across conversation sessions in a given time period. This helps developers understand their assistant’s most frequently used tasks, how users invoke them, popular task execution paths, dropout points, etc.
Conversation Insights: The Conversation Insights dashboard provides a visual user utterances map to easily discover false positives and negatives and identify opportunities to train new intents. The dashboard aims to provide a structured approach for analysts to regularly review and improve the NLU performance. Utterance clusters are formed using deep neural models, and each cluster is assigned a name representative of the underlying utterances. You can also switch to the tabular view to see the list of all intents and the corresponding clusters under these intents. You can categorize the utterances as True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN) and train them for existing or new intents.
Feedback Dashboard: The Feedback Dashboard uses the Feedback Survey feature to help you capture user feedback about your products, services, and overall experience with the virtual assistant. The in-built analytics help you get valuable insights using NPS, CSAT, or any other metrics of your choice. The Feedback Dashboard provides survey results and insights. You can see the distribution trend of feedback scores and the overall score for a given period. You can also review the full feedback results with an option to export them.
Usage Metrics: This page provides the VA activity and bot user overview. It includes both interactive and non-interactive sessions.
Containment Metrics: This page provides insights on whether queries are successfully answered by the Virtual assistant or handled by agents.