{"id":17929,"date":"2016-10-28T17:12:45","date_gmt":"2016-10-28T17:12:45","guid":{"rendered":"https:\/\/developer.kore.com\/?post_type=docs&p=8561"},"modified":"2021-03-26T05:29:02","modified_gmt":"2021-03-26T05:29:02","slug":"testing-your-bot-with-nlp","status":"publish","type":"post","link":"https:\/\/developer.kore.ai\/v8-0\/docs\/bots\/test-your-bot\/testing-your-bot-with-nlp\/","title":{"rendered":"Utterance Testing"},"content":{"rendered":"
\r\n\t
\r\n\t\t
\r\n\t\t\t

To make sure your bot responds to user utterances with related tasks, it is important that you test the bot with a variety of user inputs. Evaluating a bot with a large sample of expected user inputs not only provides insights into bot responses but also gives you a great opportunity to train the bot in interpreting diverse human expressions.<\/p>\n

You can perform all the training-related activities for a bot from the Utterance Testing<\/i>\u00a0module. We will use a sample Flight<\/i>\u00a0Booking\u00a0<\/i>bot consisting of the following tasks for use as examples across the Test and Train article.<\/p>\n

\"\"<\/a><\/p>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n

<\/div><\/div>\r\n\t
\r\n\t\t
\r\n\t\t\t

Testing the Bot<\/span><\/h2>\n

Simply put, testing a bot refers to checking if the bot can respond to a user utterance with the most relevant task. Given the flexibility of language, users will use a wide range of phrases to express the same intent.<\/p>\n

For example, you can rephrase I want to change my ticket from San Francisco to Los Angeles on Jan 1<\/em> as Please change my travel date. Can\u2019t make it on Jan 1. <\/i>The trick is to train the bot to map both of these utterances with the same intent or Modify Booking<\/i> task.<\/p>\n

So, the first step to start testing a bot is to identify a representative sample of user utterances to test the bot responses. Look for sources of data that reflect real-world usage of the language, such as support chat logs, online communities, FAQ pages of relevant portals.<\/p>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n\r\n\t

\r\n\t\t
\r\n\t\t\t

How to test the bot<\/span><\/h3>\n

Follow these steps to test a bot:<\/p>\n

    \n
  1. Open the bot that you want to test.<\/li>\n
  2. From the left navigation panel hover over Testing<\/strong> and click Utterance Testing<\/strong>.
    \n\"\"<\/a><\/li>\n
  3. In the Type a user utterance<\/b> field, enter the utterance that you want to test. Example: Rescheduling my plan. Cancel my ticket to LA.
    \n    \"\"<\/a><\/li>\n
  4. The result appears with a single, multiple, or no matching intents.
    \n    \"\"<\/a><\/li>\n<\/ol>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n
    <\/div><\/div>\r\n\t
    \r\n\t\t
    \r\n\t\t\t

    Types of Test Results<\/span><\/h3>\n

    When you test a user utterance against a bot, the NLP engine tries to find the bot tasks that match the intent. The NLP engine uses a hybrid approach using Machine Learning, Fundamental Meaning, and Knowledge Graph (if the bot has one) models to score the matching intents on relevance. The model classifies user utterances as either being Possible Matches<\/i> or Definitive Matche<\/i>s.
    \n    \"\"<\/a>
    \nDefinitive Matches<\/i> get high confidence scores and are assumed to be perfect matches for the user utterance. In published bots, if user input matches with a single Definitive Match<\/i>, the bot directly executes the task. If the utterances match with multiple Definitive Matches<\/i>, they are sent as options for the end user to choose one.<\/p>\n

    On the other hand, Possible Matches<\/i> are intents that score reasonably well against the user input but do not inspire enough confidence to be termed as exact matches. Internally the system further classifies possible matches into good and unsure matches based on their scores. If the end user utterances were generating possible matches in a published bot, the bot sends these matches as \u201cDid you mean?\u201d<\/i> suggestions for the end user.<\/p>\n

    Below are the possible outcomes of a user utterance test:<\/p>\n