GETTING STARTED
Kore.ai XO Platform
Virtual Assistants Overview
Natural Language Processing (NLP)
Concepts and Terminology
Quick Start Guide
Accessing the Platform
Navigating the Kore.ai XO Platform
Building a Virtual Assistant
Help & Learning Resources
Release Notes
Current Version
Recent Updates
Previous Versions
Deprecations
Request a Feature
CONCEPTS
Design
Storyboard
Overview
FAQs
Conversation Designer
Overview
Dialog Tasks
Mock Scenes
Dialog Tasks
Overview
Navigate Dialog Tasks
Build Dialog Tasks
Node Types
Overview
Intent Node
Dialog Node
Dynamic Intent Node
GenAI Node
GenAI Prompt
Entity Node
Form Node
Confirmation Node
Message Nodes
Logic Node
Bot Action Node
Service Node
Webhook Node
Script Node
Process Node
Agent Transfer
Node Connections
Node Connections Setup
Sub-Intent Scoping
Entity Types
Entity Rules
User Prompts or Messages
Voice Call Properties
Knowledge AI
Introduction
Knowledge Graph
Introduction
Terminology
Build a Knowledge Graph
Manage FAQs
Knowledge Extraction
Import or Export Knowledge Graph
Prepare Data for Import
Importing Knowledge Graph
Exporting Knowledge Graph
Auto-Generate Knowledge Graph
Knowledge Graph Analysis
Answer from Documents
Alert Tasks
Small Talk
Digital Skills
Overview
Digital Forms
Digital Views
Introduction
Widgets
Panels
Session and Context Variables
Context Object
Intent Discovery
Train
NLP Optimization
ML Engine
Overview
Model Validation
FM Engine
KG Engine
Traits Engine
Ranking and Resolver
Training Validations
NLP Configurations
NLP Guidelines
LLM and Generative AI
Introduction
LLM Integration
Kore.ai XO GPT Module
Prompts & Requests Library
Co-Pilot Features
Dynamic Conversations Features
Intelligence
Introduction
Event Handlers
Contextual Memory
Contextual Intents
Interruption Management
Multi-intent Detection
Amending Entities
Default Conversations
Conversation Driven Dialog Builder
Sentinment Management
Tone Analysis
Default Standard Responses
Ignore Words & Field Memory
Test & Debug
Overview
Talk to Bot
Utterance Testing
Batch Testing
Conversation Testing
Conversation Testing Overview
Create a Test Suite
Test Editor
Test Case Assertion
Test Case Execution Summary
Glossary
Health and Monitoring
NLP Health
Flow Health
Integrations
Actions
Actions Overview
Asana
Configure
Templates
Azure OpenAI
Configure
Templates
BambooHR
Configure
Templates
Bitly
Configure
Templates
Confluence
Configure
Templates
DHL
Configure
Templates
Freshdesk
Configure
Templates
Freshservice
Configure
Templates
Google Maps
Configure
Templates
Here
Configure
Templates
HubSpot
Configure
Templates
JIRA
Configure
Templates
Microsoft Graph
Configure
Templates
Open AI
Configure
Templates
Salesforce
Configure
Templates
ServiceNow
Configure
Templates
Stripe
Configure
Templates
Shopify
Configure
Templates
Twilio
Configure
Templates
Zendesk
Configure
Templates
Agents
Agent Transfer Overview
Custom (BotKit)
Drift
Genesys
Intercom
NiceInContact
NiceInContact(User Hub)
Salesforce
ServiceNow
Configure Tokyo and Lower versions
Configure Utah and Higher versions
Unblu
External NLU Adapters
Overview
Dialogflow Engine
Test and Debug
Deploy
Channels
Publishing
Versioning
Analyze
Introduction
Dashboard Filters
Overview Dashboard
Conversations Dashboard
Users Dashboard
Performance Dashboard
Custom Dashboards
Introduction
Custom Meta Tags
Create Custom Dashboard
Create Custom Dashboard Filters
LLM and Generative AI Logs
NLP Insights
Task Execution Logs
Conversations History
Conversation Flows
Conversation Insights
Feedback Analytics
Usage Metrics
Containment Metrics
Universal Bots
Introduction
Universal Bot Definition
Universal Bot Creation
Training a Universal Bot
Universal Bot Customizations
Enabling Languages
Store
Manage Assistant
Team Collaboration
Plan & Usage
Overview
Usage Plans
Templates
Support Plans
Invoices
Authorization
Conversation Sessions
Multilingual Virtual Assistants
Get Started
Supported Components & Features
Manage Languages
Manage Translation Services
Multiingual Virtual Assistant Behavior
Feedback Survey
Masking PII Details
Variables
Collections
IVR Settings
General Settings
Assistant Management
Manage Namespace
Data
Overview
Data Table
Table Views
App Definitions
Data as Service
HOW TOs
Build a Travel Planning Assistant
Travel Assistant Overview
Create a Travel Virtual Assistant
Design Conversation Skills
Create an ‘Update Booking’ Task
Create a Change Flight Task
Build a Knowledge Graph
Schedule a Smart Alert
Design Digital Skills
Configure Digital Forms
Configure Digital Views
Train the Assistant
Use Traits
Use Patterns
Manage Context Switching
Deploy the Assistant
Use Bot Functions
Use Content Variables
Use Global Variables
Use Web SDK
Build a Banking Assistant
Design Conversation Skills
Create a Sample Banking Assistant
Create a Transfer Funds Task
Create a Update Balance Task
Create a Knowledge Graph
Set Up a Smart Alert
Design Digital Skills
Configure Digital Forms
Configure Digital Views
Add Data to Data Tables
Update Data in Data Tables
Add Data from Digital Forms
Train the Assistant
Composite Entities
Use Traits
Use Patterns for Intents & Entities
Manage Context Switching
Deploy the Assistant
Configure an Agent Transfer
Use Assistant Functions
Use Content Variables
Use Global Variables
Intent Scoping using Group Node
Analyze the Assistant
Create a Custom Dashboard
Use Custom Meta Tags in Filters
Migrate External Bots
Google Dialogflow Bot
APIs & SDKs
API Reference
API Introduction
Rate Limits
API List
koreUtil Libraries
SDK Reference
SDK Introduction
Web SDK
How the Web SDK Works
SDK Security
SDK Registration
Web Socket Connect and RTM
Tutorials
Widget SDK Tutorial
Web SDK Tutorial
BotKit SDK
BotKit SDK Deployment Guide
Installing the BotKit SDK
Using the BotKit SDK
SDK Events
SDK Functions
Tutorials
BotKit - Blue Prism
BotKit - Flight Search Sample VA
BotKit - Agent Transfer
  1. Docs
  2. Virtual Assistants
  3. Analyzing Your Bot
  4. Flow Health

Flow Health

Introduction 

The Flow Health dashboard summarizes how well the flows of Virtual Assistant (VA) are performing and shows the coverage of intents and transitions in the bot definition. The Flow Health dashboard displays the overall pass or fail percentage of test cases with options to view the transitions coverage. 

The Flow Health dashboard also shows the summary of user intents that are covered and missed in test cases. You can review the uncovered intents and add relevant test cases to cover missing user intents in the test suite. 

You can also use the test result analysis to identify the issues in the dialog task definitions and fix them with actionable recommendations to improve overall NLU and Flow health.

Note: The Flow Health Dashboard is available on the XO Platform version 10.0, released on Jan 21st, 2023.

Use-Case Scenario 

In this article, the Hotel Booking virtual assistant is used as an example to understand the flow of transitions and test results analysis. The Hotel Booking VA has the Search A Hotel, Book a Hotel, Get User Data, Show User Data, and Raise Complain dialog intents with various test cases in the test suites in the Conversation Testing module.

A transition shows how a user intent is transferred from one node to another node in a dialog task. A transition is a flow of intent between two nodes. It has a From node and a To node.  For example, “Book a Hotel ” is a user intent in a dialog task. A city is an entity node in the dialog flow to which the book a hotel user intent is transitioned, and then it transitions to the state node. From the state node, it transitions to country and reservemessage nodes. Therefore, in the Book a Hotel task, you have four transitions in total. For more information, see User Intent.

Once you record a conversation flow, save it and run it in the Conversation Testing module.  The conversation testing module helps in knowing how a dialog task performs. For example, if you have a Get User Data dialog task and want to see how this dialog task is performing, you can create a Show User Data test suite with various test cases. Run the Show User Data test suite and see whether it was completed successfully, failed, paused, or not executed. For more information, see the Conversation Testing article.

The test suite results are tracked in the Conversation Testing, and the analytics for test suites are displayed in the Flow Health dashboard.  

How to Use the Flow Health Dashboard

To access the Flow Health Dashboard, follow these steps:

  1. Navigate to BUILD > Testing > Health & Monitoring and select the Flow tab.

  2. Drill down to learn how to view the Dialog Intent, FAQ, and Small Talk summary sections. In these sections, you can view the intent level summaries, along with test coverage details to easily identify the transitions or intents to be covered. You can also expand to the test case results to view the execution analytics.

Note: The test suites are used to verify the Intents, Small Talks, FAQs, and Standard Responses with test cases. The Flow Health dashboard considers standard responses in test cases to calculate overall percentages. However, the dashboard has no separate summary tile for the Standard Responses. For this reason, you will notice a slight difference in the test results’ overall count or percentage.

Flow Metrics

The Flow Health dashboard extrapolates all test suite results from the conversation testing and presents insights into your test cases with performance analytics.

View the overall percentage of the test case results in the test suites with the following flow metrics:

  • Pass – Total Number of test cases passed across all test suites.
  • Fail – Total Number of Test cases failed across all test suites.
  • Not Executed – Total Number of Test cases aborted or not executed before execution across all test suites.
  1. Click the View Test Suites link to view all the test suites that are considered in the VA with the following details:
    • Test Suite  – The name of the test suite.
    • Result – The result of the test suite, such as Passed, Failed, Aborted, or Not Executed.
    • Duration – The duration for the execution of the test suite.
    • Test Cases – The number of test cases in the test suite.
    • Pass  – The number of test cases that passed.
    • Fail – The number of test cases that failed.
    • Not Executed – The number of test cases that are not executed.
    • Pass percentage – The overall pass percentage of the test cases.
  2. Click the Test Suite Name to analyze how the test suite is performing. For example, click the FAQ_TS_Pass test suite to view the test results analysis as shown below:
  3. Select the Test Coverage tab to view the Dialog Intents, FAQs, and Small Talks covered and that are not covered in the test suite. 
  4. By default, the Dialog Intents are displayed. Click the FAQs to view FAQs details.
  5. Click the Small Talks tab to view small talk details.
  6. Click the X icon to close the Intent Details window.

Flow Dashboard Components

The key components of the Flow Health dashboard include transition coverage and intent summary panels for the test suites. The details on the available panels are explained below.

Dialog Intent Summary

The Dialog Intent Summary shows the summary of test results analysis of dialog intents in the test cases from the Conversation Testing. The Dialog Intent summary also provides insights into flow transitions that are covered and ones that are missed.

In the Dialog Intent Summary region, you can view flow transitions and test result analysis-level summaries, along with the test coverage, to easily identify intents to be covered. You can drill down to view the analytics of test results.

Note: All the test cases across test suites expecting a dialog node for assertion are considered for generating the flow summary. 

Transitions Coverage 

The Flow Health dashboard calculates the number of transitions in a dialog and shows the analysis of covered and not covered transitions. The Transition Coverage section helps you quickly understand how transitions are performing and fix issues in the specific transitions with actionable recommendations. You can analyze the transitions that are missed and add relevant test cases in the test suite to cover them in multiple iterations.

The percentage of transitions coverage is displayed as follows: 

  • Total Transitions Covered – Shows all covered transitions out to total transitions in the VA definition. 
  • Covered – Shows the number of transitions that are covered in the test cases.
  • Not Covered – Shows the percentage of total transitions that are not covered in the test cases.
    1. Click the icon beside the percentages to open the Dialog Transitions page and view the following details:
      • Task Name – The name of the dialog intent. 
      • From Node – The From node where the conversation starts to transition.
      • To Node – The To node from where the conversation is transferred to.
      • Status – The Status of the transition  – covered or not covered.
    2. Click the icon beside Not Covered percentage to view missed transitions.
    3. Create a new test suite and record missed transitions to cover them. For more information, see Conversation Testing.
    4. Click the icon to search and filter the dialog intent tasks for which you want to see the transitions coverage. You can filter transitions using the From Node, To Node, and Coverage Status filters.
      For example, in the Status Coverage column, click the to select all, covered, and not covered statuses, as shown below:
    5. Click the X icon to close the Dialog Transitions window.

Conversation Test Results Analysis

The Conversation Test Results Analysis section summarizes all test suite results. You can use the test result analysis to identify the issues in the dialog task definitions and fix them.

The percentage of test results analysis is displayed as follows: 

  • Total Test Cases – Shows the total number of test cases in a test suite.
  • Pass, Fail, or Not Executed – Shows the count and percentage of test case results that have passed, failed, or not executed.

View Intent Summary

The View Intent Summary shows how well test cases are covering and performing at the intent level, making it easier to identify the most important failing and less-covered intents.

  1. Click the View Intent Summary link to view the intent details in the Dialog Intent Details view.
  2. Once you click the View Intent Summary link, the Dialog Intent Details displays all intents that are covered in a test suite.
    This summary shows the intents covered in the test suite with the total number of transitions that are covered and missed. It also shows the percentage of test cases that have passed, failed, or not executed status against the total number of test cases for an intent.

  3. In the Dialog Intent Details window, click on Covered By for any Intent Name to view the summary. For example, the Raise Complain intent is covered in a test suite with a total of Nine transitions, out of which only Eight are covered and One is missed. The total number of test cases in this intent are Eight, out of which Seven have passed and One has failed, with Zero test case not executed.

To view the intent summary and identify the issues, follow these steps:

  1. Click the > icon to open the intent details summary for the desired intent.
  2. Click the Test Suite Name to analyze how the test suite is performing. For example, click the Dialog_Test_Raise_Complain_Fail test suite to view the test results analysis as illustrated in the screenshot below:
  3. In this test suite, the test case1 has failed because it has an incorrect data type selected for the Date field as shown below:
  4. Text assertion fails because there is a mismatch between the Expected and Actual output. For more information, see Test Case Assertion.   
  5. Click the X icon to close the Intent Details window.

FAQs Summary

The FAQ Summary section displays the FAQ Coverage and Conversation Test Results Analysis generated for all the FAQ intents in all the test cases of the selected test suite, as discussed in the View Intent Summary section.

Click View Intent Summary to get a drill-down view of the FAQ Intents that are covered or not covered in the test suites on the FAQ Intent Details window.

FAQ Intent Details Window

This window gives the drill-down view of the FAQ-level details for the selected test suite. The primary objective here is to help users know the top-performing and low-performing FAQs to identify issues proactively and work on fixing them accordingly.

For more information, see View Intent Summary section.

  • Intent Name: The name of the dialog intent.
  • Path: The node path in the Knowledge Graph.
  • Covered by (Test Suites): The count of the test suites in which the FAQ intents are covered.
  • Total Test Cases: The count of the test cases in the selected test suites for that FAQ intent.
  • Pass: The count of the passed FAQ intent test cases that passed.
  • Fail: The count of the FAQ intent test cases that failed.
  • Not Executed: The count of the FAQ intent test cases that are not executed.
  • Pass %: The overall pass percentage of the test cases.

Small Talk

The Small Talk Summary section displays the Small Talks Coverage and Conversation Test Results Analysis generated for all the Small Talk intents in the selected test suite as discussed in the View Intent Summary section.

Small Talk Intent Details Window

Click View Intent Summary to view the Small Talk Details window that gives the drill-down view of the Small Talk level details for the selected test suite. The primary objective here is to help users know the top-performing and low-performing intents to identify issues proactively and fix them accordingly.

For more information, see View Intent Summary section.

  • Intent Name: The intent name captured in the Small Talk interaction.
  • Group: The group to which the Small Talk interaction is mapped.
  • Covered by (Test Suites): The count of the test suites in which the Small Talk intent is covered.
  • Total Test Cases: The count of the test cases present in the selected test suites for that Small Talk intent.
  • Pass: The count of the Small Talk intent test cases that have passed.
  • Fail: The count of the Small Talk intent test cases that failed.
  • Not Executed: The count of the Small Talk intent test cases that are not executed.
  • Pass %: The overall pass percentage of the test cases.

Flow Health

Introduction 

The Flow Health dashboard summarizes how well the flows of Virtual Assistant (VA) are performing and shows the coverage of intents and transitions in the bot definition. The Flow Health dashboard displays the overall pass or fail percentage of test cases with options to view the transitions coverage. 

The Flow Health dashboard also shows the summary of user intents that are covered and missed in test cases. You can review the uncovered intents and add relevant test cases to cover missing user intents in the test suite. 

You can also use the test result analysis to identify the issues in the dialog task definitions and fix them with actionable recommendations to improve overall NLU and Flow health.

Note: The Flow Health Dashboard is available on the XO Platform version 10.0, released on Jan 21st, 2023.

Use-Case Scenario 

In this article, the Hotel Booking virtual assistant is used as an example to understand the flow of transitions and test results analysis. The Hotel Booking VA has the Search A Hotel, Book a Hotel, Get User Data, Show User Data, and Raise Complain dialog intents with various test cases in the test suites in the Conversation Testing module.

A transition shows how a user intent is transferred from one node to another node in a dialog task. A transition is a flow of intent between two nodes. It has a From node and a To node.  For example, “Book a Hotel ” is a user intent in a dialog task. A city is an entity node in the dialog flow to which the book a hotel user intent is transitioned, and then it transitions to the state node. From the state node, it transitions to country and reservemessage nodes. Therefore, in the Book a Hotel task, you have four transitions in total. For more information, see User Intent.

Once you record a conversation flow, save it and run it in the Conversation Testing module.  The conversation testing module helps in knowing how a dialog task performs. For example, if you have a Get User Data dialog task and want to see how this dialog task is performing, you can create a Show User Data test suite with various test cases. Run the Show User Data test suite and see whether it was completed successfully, failed, paused, or not executed. For more information, see the Conversation Testing article.

The test suite results are tracked in the Conversation Testing, and the analytics for test suites are displayed in the Flow Health dashboard.  

How to Use the Flow Health Dashboard

To access the Flow Health Dashboard, follow these steps:

  1. Navigate to BUILD > Testing > Health & Monitoring and select the Flow tab.

  2. Drill down to learn how to view the Dialog Intent, FAQ, and Small Talk summary sections. In these sections, you can view the intent level summaries, along with test coverage details to easily identify the transitions or intents to be covered. You can also expand to the test case results to view the execution analytics.

Note: The test suites are used to verify the Intents, Small Talks, FAQs, and Standard Responses with test cases. The Flow Health dashboard considers standard responses in test cases to calculate overall percentages. However, the dashboard has no separate summary tile for the Standard Responses. For this reason, you will notice a slight difference in the test results’ overall count or percentage.

Flow Metrics

The Flow Health dashboard extrapolates all test suite results from the conversation testing and presents insights into your test cases with performance analytics.

View the overall percentage of the test case results in the test suites with the following flow metrics:

  • Pass – Total Number of test cases passed across all test suites.
  • Fail – Total Number of Test cases failed across all test suites.
  • Not Executed – Total Number of Test cases aborted or not executed before execution across all test suites.
  1. Click the View Test Suites link to view all the test suites that are considered in the VA with the following details:
    • Test Suite  – The name of the test suite.
    • Result – The result of the test suite, such as Passed, Failed, Aborted, or Not Executed.
    • Duration – The duration for the execution of the test suite.
    • Test Cases – The number of test cases in the test suite.
    • Pass  – The number of test cases that passed.
    • Fail – The number of test cases that failed.
    • Not Executed – The number of test cases that are not executed.
    • Pass percentage – The overall pass percentage of the test cases.
  2. Click the Test Suite Name to analyze how the test suite is performing. For example, click the FAQ_TS_Pass test suite to view the test results analysis as shown below:
  3. Select the Test Coverage tab to view the Dialog Intents, FAQs, and Small Talks covered and that are not covered in the test suite. 
  4. By default, the Dialog Intents are displayed. Click the FAQs to view FAQs details.
  5. Click the Small Talks tab to view small talk details.
  6. Click the X icon to close the Intent Details window.

Flow Dashboard Components

The key components of the Flow Health dashboard include transition coverage and intent summary panels for the test suites. The details on the available panels are explained below.

Dialog Intent Summary

The Dialog Intent Summary shows the summary of test results analysis of dialog intents in the test cases from the Conversation Testing. The Dialog Intent summary also provides insights into flow transitions that are covered and ones that are missed.

In the Dialog Intent Summary region, you can view flow transitions and test result analysis-level summaries, along with the test coverage, to easily identify intents to be covered. You can drill down to view the analytics of test results.

Note: All the test cases across test suites expecting a dialog node for assertion are considered for generating the flow summary. 

Transitions Coverage 

The Flow Health dashboard calculates the number of transitions in a dialog and shows the analysis of covered and not covered transitions. The Transition Coverage section helps you quickly understand how transitions are performing and fix issues in the specific transitions with actionable recommendations. You can analyze the transitions that are missed and add relevant test cases in the test suite to cover them in multiple iterations.

The percentage of transitions coverage is displayed as follows: 

  • Total Transitions Covered – Shows all covered transitions out to total transitions in the VA definition. 
  • Covered – Shows the number of transitions that are covered in the test cases.
  • Not Covered – Shows the percentage of total transitions that are not covered in the test cases.
    1. Click the icon beside the percentages to open the Dialog Transitions page and view the following details:
      • Task Name – The name of the dialog intent. 
      • From Node – The From node where the conversation starts to transition.
      • To Node – The To node from where the conversation is transferred to.
      • Status – The Status of the transition  – covered or not covered.
    2. Click the icon beside Not Covered percentage to view missed transitions.
    3. Create a new test suite and record missed transitions to cover them. For more information, see Conversation Testing.
    4. Click the icon to search and filter the dialog intent tasks for which you want to see the transitions coverage. You can filter transitions using the From Node, To Node, and Coverage Status filters.
      For example, in the Status Coverage column, click the to select all, covered, and not covered statuses, as shown below:
    5. Click the X icon to close the Dialog Transitions window.

Conversation Test Results Analysis

The Conversation Test Results Analysis section summarizes all test suite results. You can use the test result analysis to identify the issues in the dialog task definitions and fix them.

The percentage of test results analysis is displayed as follows: 

  • Total Test Cases – Shows the total number of test cases in a test suite.
  • Pass, Fail, or Not Executed – Shows the count and percentage of test case results that have passed, failed, or not executed.

View Intent Summary

The View Intent Summary shows how well test cases are covering and performing at the intent level, making it easier to identify the most important failing and less-covered intents.

  1. Click the View Intent Summary link to view the intent details in the Dialog Intent Details view.
  2. Once you click the View Intent Summary link, the Dialog Intent Details displays all intents that are covered in a test suite.
    This summary shows the intents covered in the test suite with the total number of transitions that are covered and missed. It also shows the percentage of test cases that have passed, failed, or not executed status against the total number of test cases for an intent.

  3. In the Dialog Intent Details window, click on Covered By for any Intent Name to view the summary. For example, the Raise Complain intent is covered in a test suite with a total of Nine transitions, out of which only Eight are covered and One is missed. The total number of test cases in this intent are Eight, out of which Seven have passed and One has failed, with Zero test case not executed.

To view the intent summary and identify the issues, follow these steps:

  1. Click the > icon to open the intent details summary for the desired intent.
  2. Click the Test Suite Name to analyze how the test suite is performing. For example, click the Dialog_Test_Raise_Complain_Fail test suite to view the test results analysis as illustrated in the screenshot below:
  3. In this test suite, the test case1 has failed because it has an incorrect data type selected for the Date field as shown below:
  4. Text assertion fails because there is a mismatch between the Expected and Actual output. For more information, see Test Case Assertion.   
  5. Click the X icon to close the Intent Details window.

FAQs Summary

The FAQ Summary section displays the FAQ Coverage and Conversation Test Results Analysis generated for all the FAQ intents in all the test cases of the selected test suite, as discussed in the View Intent Summary section.

Click View Intent Summary to get a drill-down view of the FAQ Intents that are covered or not covered in the test suites on the FAQ Intent Details window.

FAQ Intent Details Window

This window gives the drill-down view of the FAQ-level details for the selected test suite. The primary objective here is to help users know the top-performing and low-performing FAQs to identify issues proactively and work on fixing them accordingly.

For more information, see View Intent Summary section.

  • Intent Name: The name of the dialog intent.
  • Path: The node path in the Knowledge Graph.
  • Covered by (Test Suites): The count of the test suites in which the FAQ intents are covered.
  • Total Test Cases: The count of the test cases in the selected test suites for that FAQ intent.
  • Pass: The count of the passed FAQ intent test cases that passed.
  • Fail: The count of the FAQ intent test cases that failed.
  • Not Executed: The count of the FAQ intent test cases that are not executed.
  • Pass %: The overall pass percentage of the test cases.

Small Talk

The Small Talk Summary section displays the Small Talks Coverage and Conversation Test Results Analysis generated for all the Small Talk intents in the selected test suite as discussed in the View Intent Summary section.

Small Talk Intent Details Window

Click View Intent Summary to view the Small Talk Details window that gives the drill-down view of the Small Talk level details for the selected test suite. The primary objective here is to help users know the top-performing and low-performing intents to identify issues proactively and fix them accordingly.

For more information, see View Intent Summary section.

  • Intent Name: The intent name captured in the Small Talk interaction.
  • Group: The group to which the Small Talk interaction is mapped.
  • Covered by (Test Suites): The count of the test suites in which the Small Talk intent is covered.
  • Total Test Cases: The count of the test cases present in the selected test suites for that Small Talk intent.
  • Pass: The count of the Small Talk intent test cases that have passed.
  • Fail: The count of the Small Talk intent test cases that failed.
  • Not Executed: The count of the Small Talk intent test cases that are not executed.
  • Pass %: The overall pass percentage of the test cases.
메뉴