{"id":18020,"date":"2017-12-28T09:02:42","date_gmt":"2017-12-28T09:02:42","guid":{"rendered":"http:\/\/developer.kore.ai\/v10-3\/?post_type=docs&#038;p=12382"},"modified":"2024-01-22T05:14:22","modified_gmt":"2024-01-22T05:14:22","slug":"batch-testing","status":"publish","type":"post","link":"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/test-your-bot\/batch-testing\/batch-testing\/","title":{"rendered":"Batch Testing"},"content":{"rendered":"<section class=\"l-section wpb_row height_auto\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row type_default valign_top\"><div class=\"vc_col-sm-12 wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\r\n\t<div class=\"wpb_text_column \">\r\n\t\t<div class=\"wpb_wrapper\">\r\n\t\t\t<p>Once you have built and trained your bot, the most important question that arises is how good is your bot\u2019s learning model? So, evaluating your bot\u2019s performance is important to delineate how good your bot understands the user utterances.<\/p>\n<p>The <strong>Batch Testing<\/strong> feature helps you discern the ability of your bot to correctly recognize the expected intents and entities from a given set of utterances. This involves the execution of a series of tests to get a detailed statistical analysis and gauge the performance of your bot&#8217;s ML model.<\/p>\n<p><b>Note<\/b>: The Batch Testing dashboard displays the summary of test coverage, performance, and training recommendations for multiple test suites based on validations of the intent identification capabilities of your NLU model. The <b>Health and Monitoring<\/b> dashboard is a one-stop place that displays this information for the individual test suite(s) the user selects. <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/analyzing-your-bot\/virtual-assistants-health-and-monitoring\/\">Learn More<\/a>.<\/p>\n<p>To conduct a batch test, you can use predefined test suites available in the builder or create your own custom test suites. Based on your requirement, the test suites can be run to view the desired results. This option can be accessed from the <strong>Testing -&gt; Batch Testing<\/strong> option from the left navigation menu.<\/p>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n<div class=\"w-separator size_small with_line width_default thick_1 style_solid color_border align_center\"><div class=\"w-separator-h\"><\/div><\/div>\r\n\t<div class=\"wpb_text_column \">\r\n\t\t<div class=\"wpb_wrapper\">\r\n\t\t\t<h1><span class=\"ez-toc-section\" id=\"Best_Practices\">Best Practices<\/span><\/h1>\n<ul>\n<li>An optimal approach to bot NLP training is to first create a test suite of most of the use cases(user utterances) that the bot needs to identify, run it against the model and start training for the ones that failed.<\/li>\n<li>Create\/update batch testing modules for high usage utterances.<\/li>\n<li>Publish the trained model only after detailed testing.<\/li>\n<li>When naming the intent, ensure that the name is relatively short (3-5 words) and does not have special characters or words from the Stop Wordlist. Try to ensure the intent name is close to what the users request in their utterance.<\/li>\n<li>Batch Test executions do not consider the context of the user. Hence you might see some False Negatives in the test results which in fact are True Positives in the actual bot when the context is taken into consideration.<\/li>\n<li>The \u2018count\u2019 in Batch Test results refers to the \u2018unique assertion\u2019 statements and not necessarily the number of the rows in the CSV file. Batch tests use a rule to validate the unique assertions based on the \u2018utterance\u2019 in consecutive rows. If two consecutive rows have same utterances and different entity values, the XO platform considers it as one assertion statement which accepts both the entity values.<\/li>\n<\/ul>\n<h1><span class=\"ez-toc-section\" id=\"Managing_Test_Suites\">Managing Test Suites<\/span><\/h1>\n<p>Kore.ai provides a few out-of-the-box <b>Test Suites<\/b> to perform batch testing. \u2018<b>Developer defined utterances\u2019<\/b> and \u2018<b>Successful user utterances\u2019 <\/b>are the built-in test suites that can be run to perform Batch Testing. You can also create a <b>New Test Suite<\/b> for testing a custom set of utterances.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Developer_defined_utterances\">Developer defined utterances<\/span><\/h2>\n<p>This test suite validates the utterances that have been previously added and trained by the developer from Machine Learning Utterances screen. Using this test suite would mean testing collectively the entire set of utterances that a developer has added for all tasks of the bot.<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725163310\/developer-du.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-43512\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725163310\/developer-du.png\" alt=\"\" width=\"700\" height=\"180\" \/><\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Successful_user_utterances\">Successful user utterances<\/span><\/h2>\n<p>This test suite includes all the end-user utterances that have successfully matched one or more intents and the corresponding task is fully executed. You can also find these utterances from the &#8216;Intent found&#8217; section of the Analyze module.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725163539\/successful-utterances.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-43515\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725163539\/successful-utterances.png\" alt=\"\" width=\"700\" height=\"180\" \/><\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Adding_a_New_Test_Suite\">Adding a New Test Suite<\/span><\/h2>\n<p>To add a new test suite, follow the steps below:<\/p>\n<ol>\n<li style=\"list-style-type: none;\">\n<ol>\n<li>Go to <b>Build<\/b> &gt; <b>Testing<\/b> &gt; <b>Batch Testing<\/b>, and click <strong>New Test Suite<\/strong>.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085811\/2023-03-14_12-17-05.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-56116\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085811\/2023-03-14_12-17-05.png\" alt=\"\" width=\"700\" height=\"345\" \/><\/a><\/p>\n<ol>\n<li>In the <strong>New test Suite<\/strong> window, add a <strong>name <\/strong><span style=\"font-weight: 400;\">and provide a <\/span><strong>description<\/strong><span style=\"font-weight: 400;\">.\u00a0<\/span><\/li>\n<li>Next, choose how to add test cases: either manually or by uploading a test case file.<\/li>\n<\/ol>\n<ul>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Selecting <\/span><strong>Add Manually<\/strong><span style=\"font-weight: 400;\"> lets you add test cases manually or use LLM and Generative AI to generate test cases automatically.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Selecting <\/span><strong>Upload Test Cases File<\/strong><span style=\"font-weight: 400;\"> lets you upload a file containing test cases.<\/span><br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090506\/newtest.png\"><img loading=\"lazy\" decoding=\"async\" class=\" size-full wp-image-56121\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090506\/newtest.png\" alt=\"\" width=\"500\" height=\"504\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090506\/newtest.png 833w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090506\/newtest-150x150.png 150w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090506\/newtest-298x300.png 298w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090506\/newtest-100x100.png 100w\" sizes=\"auto, (max-width: 500px) 100vw, 500px\" \/><\/a><\/li>\n<\/ul>\n<h3><span class=\"ez-toc-section\" id=\"Adding_Test_Cases_Manually\"><span style=\"font-weight: 400;\">Adding Test Cases Manually<\/span><\/span><\/h3>\n<p>Once you create a test suite to which you have chosen to add test cases manually, you can either add them manually or generate them automatically.<\/p>\n<p>In the Test Cases page, click the <strong>+Add Test Case<\/strong>.<\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085732\/add-manually.png\"><img loading=\"lazy\" decoding=\"async\" class=\" size-full wp-image-56092\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085732\/add-manually.png\" alt=\"\" width=\"700\" height=\"337\" \/><\/a><\/p>\n<p><span style=\"font-weight: 400;\">In the <\/span><strong>Add Test Case<\/strong><span style=\"font-weight: 400;\"> window, select or enter information in the following fields:<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">The <\/span><strong>Intent <\/strong><span style=\"font-weight: 400;\">that corresponds to the Dialog Task you want to test. You can add\/tag one or more intents to a test utterance.<br \/>\nYou can add up to three intents (Dialog, FAQ, or Small Talk) for a test utterance. It helps in scenarios where ambiguity is by-design and should be considered as True Positive. For example, for the utterance &#8220;I need help to book a flight ticket,&#8221; the expected intent can be either &#8220;Book flight&#8221; (Dialog) or &#8220;How to book a flight?&#8221; (FAQ).<br \/>\n<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">The <\/span><strong>Parent Intent<\/strong><span style=\"font-weight: 400;\"> within that task.<\/span><\/li>\n<li style=\"font-weight: 400;\"><strong>Test Utterances<\/strong><span style=\"font-weight: 400;\">: You can add multiple utterances, each on a new line.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">The <\/span><strong>Entity Order<\/strong><span style=\"font-weight: 400;\">: Select one from the available options, depending on the intent you are working with. You can only select one item. The Entity Order selection is not available if you have added multiple intents.<\/span><br \/>\n<table style=\"width: 100%;\">\n<tbody>\n<tr>\n<td style=\"width: 50%;\" align=\"left\"><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230627152228\/one-intent-for-a-test-utterance.png\"><img decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230627152228\/one-intent-for-a-test-utterance.png\" alt=\"\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><em>One Intent for a Test Utterance<\/em><\/p>\n<\/td>\n<td style=\"width: 50%;\" align=\"right\"><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230627152321\/multiple-intents-for-a-test-utterance.png\"><img decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230627152321\/multiple-intents-for-a-test-utterance.png\" alt=\"\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><em>Multiple Intents for a Test Utterance<\/em><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n<li style=\"font-weight: 400;\">Click <strong>Save<\/strong> when ready.<\/li>\n<li>Once added, your test cases are listed and ready to run. You can use the <strong>Add Manually<\/strong> button at the top of the list to add more test cases. The option to Generate Test cases only displays if you have enabled LLM and Generative AI for the VA.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090816\/add-new-case.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-56125\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090816\/add-new-case.png\" alt=\"\" width=\"1917\" height=\"151\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090816\/add-new-case.png 1917w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090816\/add-new-case-300x24.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320090816\/add-new-case-1024x81.png 1024w\" sizes=\"auto, (max-width: 1917px) 100vw, 1917px\" \/><\/a><\/li>\n<\/ol>\n<h3><span class=\"ez-toc-section\" id=\"Generating_Test_Cases_Automatically\"><span style=\"font-weight: 400;\">Generating Test Cases Automatically<\/span><\/span><\/h3>\n<p><span style=\"font-weight: 400;\">You can automatically generate batch test cases if you have enabled <\/span><a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/nlp\/llm-and-generative-ai\/\"><span style=\"font-weight: 400;\">LLM and Generative AI <\/span><\/a><span style=\"font-weight: 400;\">for your Virtual Assistant. You can access the feature either right after creating a new test case, as shown above, or from the test cases list.<\/span><\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085753\/goto-generate-test-cases.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-56104\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085753\/goto-generate-test-cases.png\" alt=\"\" width=\"700\" height=\"265\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085753\/goto-generate-test-cases.png 1980w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085753\/goto-generate-test-cases-300x114.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085753\/goto-generate-test-cases-1024x388.png 1024w\" sizes=\"auto, (max-width: 700px) 100vw, 700px\" \/><\/a><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Steps_to_Generate_Test_Cases\">Steps to Generate Test Cases<\/span><\/h3>\n<p>To generate test cases automatically, follow the steps below:<\/p>\n<ol>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Click <\/span><strong>Generate Test Cases<\/strong><span style=\"font-weight: 400;\"> from any of the two areas mentioned above.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">In the <\/span><strong>Generate Test Case<\/strong><span style=\"font-weight: 400;\"> window, select the <\/span><strong>Dialog Task<\/strong><span style=\"font-weight: 400;\"> you want to test.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Click <\/span><strong>Generate<\/strong><span style=\"font-weight: 400;\">. Wait for a few moments until the generation completes.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085737\/generate-add-intent.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-56095\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085737\/generate-add-intent.png\" alt=\"\" width=\"649\" height=\"275\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085737\/generate-add-intent.png 1920w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085737\/generate-add-intent-300x127.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085737\/generate-add-intent-1024x434.png 1024w\" sizes=\"auto, (max-width: 649px) 100vw, 649px\" \/><\/a><\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Once test cases are generated, you can reject some of them. If required, click Generate more to get more suggestions.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Click <\/span><strong>Add Test Cases<\/strong><span style=\"font-weight: 400;\"> when ready.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085742\/generation-done.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-56098\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085742\/generation-done.png\" alt=\"\" width=\"649\" height=\"289\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085742\/generation-done.png 1920w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085742\/generation-done-300x133.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085742\/generation-done-1024x455.png 1024w\" sizes=\"auto, (max-width: 649px) 100vw, 649px\" \/><\/a><br \/>\n<\/span><\/li>\n<\/ol>\n<p><span style=\"font-weight: 400;\">Once you add the test cases, they are listed within your test suite, along with any other cases you might have added manually or generated in the past. You can continue adding test cases to the suite using these two methods.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\"><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085805\/test-cases-added.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-56113\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085805\/test-cases-added.png\" alt=\"\" width=\"650\" height=\"289\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085805\/test-cases-added.png 1920w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085805\/test-cases-added-300x133.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230320085805\/test-cases-added-1024x455.png 1024w\" sizes=\"auto, (max-width: 650px) 100vw, 650px\" \/><\/a><\/span><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Adding_a_Test_Suite_by_Uploading_a_Test_Cases_File\">Adding a Test Suite by Uploading a Test Cases File<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">If you <\/span><strong>Upload a Test Cases File<\/strong><span style=\"font-weight: 400;\">, you can import an array of test utterances, also known as a Dataset at once in a batch file. The Dataset file must in a CSV or JSON format and can have a maximum of 1000 utterances. You can download the sample CSV or JSON file formats while creating the test suite.<br \/>\n<\/span><\/p>\n<div class=\"alert alert-info\">\n<p><b>Important Tip<\/b>: For a Universal Bot, to create a test suite, you can add multiple intents in the CSV\/JSON file using the upload method. The format includes the intents and the corresponding bot names as follows:<\/p>\n<pre class=\"code-limit-200px\">{\r\n\u201cinput\u201d: \u201cUtterance\u201d,\r\n\u201cintent\u201d: \u201cIntent 1|Intent 2|Intent 3\u201d,\r\n\u201cbotName\u201d: \u201cBot 1|Bot 2|Bot 3\u201d\r\n}\r\n<\/pre>\n<p><b>Example<\/b><\/p>\n<pre class=\"code-limit-200px\">{\r\n\u201cinput\u201d: \u201cNeed to check the multibot function\u201d,\r\n\u201cintent\u201d: \u201cAge Predictor|IVRnode|Check Rephase\u201d,\r\n\u201cbotName\u201d: \u201cCheck Bot|XOP211|Check Rephase\u201d\r\n}\r\n<\/pre>\n<\/div>\n<p>The ability to create test suites manually is not yet available for universal bots.<\/p>\n<p><b>JSON Format for Test Suite<\/b><\/p>\n<p>The JSON format for creating custom suites allows you to define an array of test cases where each test case should consist of an utterance to be tested, the intent against which the utterance to be tested, and optionally define the list of expected entities to be determined from the utterance. If expected intent is a child intent, then you can also include the parent intent to be considered.<\/p>\n<p>The sample code is given below:<\/p>\n<pre class=\"code-limit-400px\">{\r\n    \"testCases\": [\r\n    {\r\n      \"input\": \"Send 200 dollars to Leonardo\",\r\n      \"intent\": \"Transfer Funds\",\r\n      \"entities\": [\r\n        {\r\n          \"entityValue\": \"200 USD\",\r\n          \"entityName\": \"TransferAmount\"\r\n        },\r\n        {\r\n          \"entityValue\": \"Leonardo\",\r\n          \"entityName\": \"PayeeName\"\r\n        }\r\n      ],\r\n      \"entityOrder\":[\"TransferAmount\", \"PayeeName\"]\r\n    },\r\n    {\r\n      \"input\": \"What is the balance in my checking account\",\r\n      \"intent\": \"Show Balance\",\r\n      \"parentIntent\": \"Transfer Funds\"\r\n    },\r\n    {\r\n      \"input\": \"Repeat this transfer every month\",\r\n      \"intent\": \"Setup Auto Pay\",\r\n      \"parentIntent\": \"Transfer Funds\"\r\n    },\r\n    {\r\n      \"input\": \"Show my past 20 transactions\",\r\n      \"intent\": \"Show Account Statement\",\r\n      \"entities\": [\r\n        {\r\n          \"entityValue\": \"20\",\r\n          \"entityName\": \"HistorySize\"\r\n        }\r\n      ]\r\n    },\r\n    {\r\n      \"input\": \"Pay my credit card dues\",\r\n      \"intent\": \"Pay Bill\"\r\n    },\r\n\t{\r\n      \"input\": \"I need to pay my monthly credit card bill\",\r\n      \"intent\": \"Pay Bill | Setup Auto Pay | Transfer Funds\"\r\n    },\r\n\t{\r\n\t\r\n      \"input\": \"Open an account for me\",\r\n      \"intent\": \"Open Account | How to Open Account?\"\r\n    },\r\n    {\r\n      \"input\": \"looks like something is wrong in my statement\",\r\n      \"intent\":\"trait: Account Statement||Issue||Account Type\"\r\n    },\r\n    {\r\n      \"input\": \"How are you doing today\",\r\n      \"intent\":\"emohowzit\"\r\n    },\r\n    {\r\n      \"input\": \"Thank you\",\r\n      \"intent\":\"thank~you\"\r\n    },\r\n    {\r\n      \"input\": \"I understand\",\r\n      \"intent\":\"i_understand\"\r\n    },\r\n    {\r\n      \"input\": \"That's great\",\r\n      \"intent\":\"~positiveackwords\"\r\n    },\r\n    {\r\n      \"input\": \"How many people can you speak to at once?\",\r\n      \"intent\":\"How many people can you speak to at once?\"\r\n    },\r\n    {\r\n      \"input\": \"That's sweet\",\r\n      \"intent\":\"That's sweet\",\r\n      \"parentIntent\": \"How many people can you speak to at once?\"\r\n    },\r\n    {\r\n      \"input\": \"What else can you do?\",\r\n      \"intent\":\" What else can you do?\",\r\n      \"parentIntent\": \"How many people can you speak to at once? &gt;&gt; That's sweet \"\r\n    },\r\n    {\r\n      \"input\": \"What else can you do?\",\r\n      \"intent\":\"What else\"\r\n    },\r\n    {\r\n      \"input\": \"What's your age?\",\r\n      \"intent\":\"How old are you?\"\r\n    },\r\n    {\r\n      \"input\": \"Who created you?\",\r\n      \"intent\":\"Who made you?\"\r\n    },\r\n    {\r\n      \"input\": \"How many people talk to you?\",\r\n      \"intent\":\"How many people can you speak to at once?\"\r\n    },\r\n    {\r\n      \"input\": \"What's your name?\",\r\n      \"intent\":\"What is your name?\"\r\n    },\r\n    {\r\n      \"input\": \"What are you called?\",\r\n      \"intent\":\"What ~you called?\"\r\n    }\r\n  ]\r\n}\r\n<\/pre>\n<ul>\n<li>For Entities that have the Multi-Item enabled, values need to be given as: <code>entity1||entity2<\/code><\/li>\n<li>Composite Entities require passing values in the following format: <code>component1name:entityValue|component2name:entityValue2<\/code>The sample code is given below:\n<pre class=\"code-limit-400px\">{\r\n  \"testCases\": [\r\n    {\r\n      \"input\": \"Add cart to Apples , Grapes\",\r\n      \"intent\": \"Add to cart\",\r\n      \"entities\": [\r\n        {\r\n          \"entityValue\": \"Apples||Grapes\",\r\n          \"entityName\": \"LOV\"\r\n        }\r\n       ]\r\n     },\r\n      {\r\n         \"input\": \"I booked a ticket from Hyderabad on July 6th, 2018 and it costed me Rs. 1200\",\r\n         \"intent\": \"Book a flight\",\r\n         \u201centities\u201d: [\r\n           {\r\n            \"entityValue\": \"City:Hyderabad|Date:2018-07-06}Curr:1200 INR\",\r\n            \"entityName\": \"Composite5\",      \r\n           }\r\n        ]\r\n      }\r\n   }\r\n<\/pre>\n<\/li>\n<li>The order in which the entities are to be extracted can be given as: <code>\"entityOrder\":[\"TransferAmount\", \"PayeeName\"]<\/code>. If the order is not provided or partially provided, the platform determines the shortest route covering all the entities as the default order.<\/li>\n<\/ul>\n<table>\n<tbody>\n<tr>\n<th>Property Name<\/th>\n<th>Type<\/th>\n<th>Description<\/th>\n<\/tr>\n<tr>\n<td>Test Cases<\/td>\n<td>Array<\/td>\n<td>Consists of the following:<\/p>\n<ul>\n<li>input<\/li>\n<li>intent<\/li>\n<li>entities<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>input<\/td>\n<td>String<\/td>\n<td>End-user Utterance. Note that if any of the utterances are beyond 3000 characters, the file upload would be rejected.<\/td>\n<\/tr>\n<tr>\n<td>intent<\/td>\n<td>String<\/td>\n<td>Determine the objective of an end-user utterance (can be task name or primary question in case of FAQ test case)<\/p>\n<p>Post-release 7.3, this property can be used to define traits to be identified against this utterance by using the prefix &#8220;<em>trait<\/em>&#8221; for example, <em>Trait: Trait Name1|| Trait Name2||Trait Name3<\/em><\/p>\n<p>Post-release 8.0, this property can include the expected Small Talk pattern.<\/td>\n<\/tr>\n<tr>\n<td>entities<\/td>\n<td>Array [Optional]<\/td>\n<td>Consists of an array of entities to be determined from the input sentence:<\/p>\n<ul>\n<li>entityValue<\/li>\n<li>entityName<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>entityValue<\/td>\n<td>String<\/td>\n<td>Value of the entity expected to be determined from the utterance. You can define the expected Entity Value as a string or use a Regular Expression. For the purpose of Batch Testing, the platform flattens all entity values into string formats. Please refer to the <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/test-your-bot\/batch-testing\/batch-testing\/#Entity_Format_Conversions\">Entity Format Conversions<\/a> section for more information.<\/td>\n<\/tr>\n<tr>\n<td>entityName<\/td>\n<td>String<\/td>\n<td>Name of the entity expected to be determined from the utterance<\/td>\n<\/tr>\n<tr>\n<td>entityOrder<\/p>\n<p>(ver7.1 onwards)<\/td>\n<td>Array [Optional]<\/td>\n<td>An array of entity names specifying the order in which the entities are to be extracted.<\/p>\n<p>If the order is not provided or partially provided, the platform determines the shortest route covering all the entities as the default order.<\/td>\n<\/tr>\n<tr>\n<td>parentIntent<\/td>\n<td>String [Optional]<\/td>\n<td>Define parent intent to be considered if the intent is a sub-intent.<br \/>\nIn the case of Small Talk, this field should be populated when the Small Talk is contextual follow-up intent; in case of multi-level contextual intent the parent intents should be separated by the delimiter ||<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><b>CSV Format for Test Suite<\/b><\/p>\n<p>CSV format for creating custom suites allows you to define test cases as records in CSV file where each test case should consist of an utterance to be tested, the intent against which the utterance to be tested, and optionally define entities to be determined from the utterance. If your test case requires more than one entity to be detected from a sentence, then you have to include an extra row for each of the additional entities to be detected. If expected intent is a child intent, then you can also include the parent intent to be considered.<\/p>\n<pre class=\"code-limit-200px\">input,intent,parentIntent,entityName,entityValue,entityOrder\r\nSend 200 dollars to Leonardo, Transfer Funds,,TransferAmount,200 USD,,\r\n,,,PayeeName,Leonardo, TransferAmount&gt;PayeeName\r\nWhat is the balance in my checking account,Show Balance, Transfer Funds,,,\r\nRepeat this transfer every month, Setup Auto Pay, Transfer Funds,,,\r\nShow my past 20 transactions, Show Account Statement,,HistorySize,20,,\r\nPay my credit card dues,Pay Bill,,,,,\r\n<\/pre>\n<ul>\n<li>For Entities that have the Multi-Item enabled values need to be given as: <code>entity1||entity2<\/code><\/li>\n<li>Composite Entities require the values in the following format:<code>component1name:entityValue|component2name:entityValue2<\/code>\n<pre class=\"code-limit-200px\">input,intent,parentIntent,entityName,entityValue\r\n\u201cAdd cart to Apples , Grapes\u201d,Add to cart,LOV,Apples||Grapes,,\r\n\u201cI booked a ticket from hyderabad on July 6th, 2018 and it cost me Rs. 1200\u201d,Composite Test,Composite55,City5:Hyderabad|Date5:2018-07-06\u201d|Curr5:1200 INR\r\n<\/pre>\n<\/li>\n<li>The order of extraction of entity value can be mentioned in the following format: <code>entity3&gt;entity4&gt;entity1<\/code>. If the order is not provided or partially provided, the platform determines the shortest route covering all the entities as the default order.<\/li>\n<\/ul>\n<table>\n<tbody>\n<tr>\n<th>Column Name<\/th>\n<th>Type<\/th>\n<th>Description<\/th>\n<\/tr>\n<tr>\n<td>input<\/td>\n<td>String<\/td>\n<td>Utterance given by the end-user. Note that if any of the utterances are beyond 3000 characters, the file upload would be rejected.<\/td>\n<\/tr>\n<tr>\n<td>intent<\/td>\n<td>String<\/td>\n<td>Determine the objective of an end-user utterance (can be task name or primary question in case of FAQ test case)<\/p>\n<p>Post release 7.3, this property can be used to define traits to be identified against this utterance by using the prefix &#8220;<em>trait<\/em>&#8221; for example, <em>Trait: Trait Name1|| Trait Name2||Trait Name3<\/em><\/p>\n<p>Post-release 8.0, this property can include the expected Small Talk pattern.<\/td>\n<\/tr>\n<tr>\n<td>parentIntent<\/td>\n<td>String [Optional]<\/td>\n<td>Define parent intent to be considered if the intent is a sub-intent<br \/>\nIn the case of Small Talk, this field should be populated when the Small Talk is contextual follow-up intent and the intent would be matched assuming that the follow-up intent criteria is met; in the case of multi-level contextual intent the parent intents should be separated by the delimiter ||<\/td>\n<\/tr>\n<tr>\n<td>entityValue<\/td>\n<td>String [Optional]<\/td>\n<td>Value of the entity expected to be determined from the utterance. You can define the expected Entity Value as a string or use a Regular Expression. For the purpose of Batch Testing, the platform flattens all entity values into string formats. Please refer to the <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/test-your-bot\/batch-testing\/batch-testing\/#Entity_Format_Conversions\">Entity Format Conversions<\/a> section for more information.<\/td>\n<\/tr>\n<tr>\n<td>entityName<\/td>\n<td>String [Optional]<\/td>\n<td>Name of the entity expected to be determined from the utterance<\/td>\n<\/tr>\n<tr>\n<td>entityOrder<\/p>\n<p>(ver7.1 onwards)<\/td>\n<td>Array [Optional]<\/td>\n<td>An array of entity names separated by &gt; specifying the order in which the entities are to be extracted.<\/p>\n<p>If the order is not provided or partially provided, the platform defines the implicit order to process first the NER and pattern entities and then the remaining entities.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h3><span class=\"ez-toc-section\" id=\"Entity_Format_Conversions\">Entity Format Conversions<\/span><\/h3>\n<table>\n<tbody>\n<tr>\n<th>Entity Type<\/th>\n<th>Sample Entity ValueType<\/th>\n<th>Value in Flat Format<\/th>\n<th>Order of Keys<\/th>\n<\/tr>\n<tr>\n<td>Address<\/td>\n<td>P.O. Box 3700 Eureka, CA 95502<\/td>\n<td>P.O. Box 3700 Eureka, CA 95502<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Airport<\/td>\n<td>{ &#8220;IATA&#8221;: &#8220;IAD&#8221;, &#8220;AirportName&#8221;: &#8220;Washington Dulles International Airport&#8221;, &#8220;City&#8221;: &#8220;Washington D.C.&#8221;, &#8220;CityLocal&#8221;: &#8220;Washington&#8221;, &#8220;ICAO&#8221;: &#8220;KIAD&#8221;, &#8220;Latitude&#8221;: &#8220;38.94&#8221;, &#8220;Longitude&#8221;: &#8220;-77.46&#8221; }<\/td>\n<td>Washington Dulles International Airport IAD KIAD 38.94 -77.46 Washington D.C. Washington<\/td>\n<td>AirportName IATA ICAO Latitude Longitude City CityLocal<\/td>\n<\/tr>\n<tr>\n<td>City<\/td>\n<td>Washington<\/td>\n<td>Washington<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Country<\/td>\n<td>{ &#8220;alpha3&#8221;: &#8220;IND&#8221;, &#8220;alpha2&#8221;: &#8220;IN&#8221;, &#8220;localName&#8221;: &#8220;India&#8221;, &#8220;shortName&#8221;: &#8220;India&#8221;, &#8220;numericalCode&#8221;: 356}<\/td>\n<td>IN IND 356 India India<\/td>\n<td>alpha2 alpha3 numericalCode localName shortName<\/td>\n<\/tr>\n<tr>\n<td>Company or Organization Name<\/td>\n<td>Kore.ai<\/td>\n<td>Kore.ai<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Color<\/td>\n<td>Blue<\/td>\n<td>Blue<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Currency<\/td>\n<td>[{ &#8220;code&#8221;: &#8220;USD&#8221;, &#8220;amount&#8221;: 10 }]<\/td>\n<td>10 USD<\/td>\n<td>amount code<\/td>\n<\/tr>\n<tr>\n<td>Date<\/td>\n<td>2018-10-25<\/td>\n<td>2018-10-25<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Date Period<\/td>\n<td>{ &#8220;fromDate&#8221;: &#8220;2018-11-01&#8221;, &#8220;toDate&#8221;: &#8220;2018-11-30&#8221; }<\/td>\n<td>2018-11-01 2018-11-30<\/td>\n<td>fromDate toDate<\/td>\n<\/tr>\n<tr>\n<td>Date Time<\/td>\n<td>2018-10-24T13:03:03+05:30<\/td>\n<td>2018-10-24T13:03:03+05:30<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Description<\/td>\n<td>Sample Description<\/td>\n<td>Sample Description<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Email<\/td>\n<td>user1@emaildomain.com<\/td>\n<td>user1@emaildomain.com<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>List of Items(Enumerated)<\/td>\n<td>Apple<\/td>\n<td>Apple<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>List of Items(Lookup)<\/td>\n<td>Apple<\/td>\n<td>Apple<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Location<\/td>\n<td>{ &#8220;formatted_address&#8221;: &#8220;8529 Southpark Cir #100, Orlando, FL 32819, USA&#8221;, &#8220;lat&#8221;: 28.439148,&#8221;lng&#8221;: -81.423733 }<\/td>\n<td>8529 Southpark Cir #100, Orlando, FL 32819, USA 28.439148 -81.423733<\/td>\n<td>formatted_address lat lng<\/td>\n<\/tr>\n<tr>\n<td>Number<\/td>\n<td>100<\/td>\n<td>100<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Person Name<\/td>\n<td>Peter Pan<\/td>\n<td>Peter Pan<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Percentage<\/td>\n<td>0.25<\/td>\n<td>0.25<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Phone Number<\/td>\n<td>+914042528888<\/td>\n<td>+914042528888<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Quantity<\/td>\n<td>{ &#8220;unit&#8221;: &#8220;meter&#8221;, &#8220;amount&#8221;: 16093.4, &#8220;type&#8221;: &#8220;length&#8221;, &#8220;source&#8221;: &#8220;10 miles&#8221; }<\/td>\n<td>16093.4 meter length 10 miles<\/td>\n<td>amount unit type source<\/td>\n<\/tr>\n<tr>\n<td>String<\/td>\n<td>Sample String<\/td>\n<td>Sample String<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Time<\/td>\n<td>T13:15:55+05:30<\/td>\n<td>T13:15:55+05:30<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Time Zone<\/td>\n<td>-04:00<\/td>\n<td>-04:00<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>URL<\/td>\n<td>https:\/\/kore.ai<\/td>\n<td>https:\/\/kore.ai<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Zip Code<\/td>\n<td>32819<\/td>\n<td>32819<\/td>\n<td><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h4>Importing a Dataset file<\/h4>\n<ol>\n<li>Click <b>New Test Suite<\/b> on the batch testing page. A dialog box to import the dataset appears.<\/li>\n<li>Enter a <b>Name, Description, <\/b>and choose a <b>Dataset Type<\/b> in the respective boxes for your dataset file.<\/li>\n<li>To import the Dataset file, click <b>Choose File <\/b>to locate and select a JSON or CSV file containing the utterances as per the Dataset Type selected.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725170312\/new-test-suite-1-2.png\"><br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-43530\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725170312\/new-test-suite-1-2.png\" alt=\"\" width=\"400\" height=\"380\" \/><\/a><\/li>\n<li>Click <b>Create<\/b>. The dataset file is displayed as an option to run the test suite on the Batch Testing page:<\/li>\n<\/ol>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n<div class=\"w-separator size_small with_line width_default thick_1 style_solid color_border align_center\"><div class=\"w-separator-h\"><\/div><\/div>\r\n\t<div class=\"wpb_text_column \">\r\n\t\t<div class=\"wpb_wrapper\">\r\n\t\t\t<h2><span class=\"ez-toc-section\" id=\"Running_Test_Suites\">Running Test Suites<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">The following steps guide you on how to run a batch test on your bot and get a detailed analytical report on the utterances based on the test results. To get started, go to Build &gt; Testing &gt; <\/span><b>Batch Testing<\/b><span style=\"font-weight: 400;\">.<\/span><\/p>\n<div class=\"alert alert-info\"><b>Note:<\/b> Before testing, it is essential to add and train your bot with a considerable number of utterances using Machine Learning.<\/div>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725165748\/batch-testing-menu.png\"><br \/>\n<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-43526\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725165748\/batch-testing-menu.png\" alt=\"\" width=\"219\" height=\"461\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725165748\/batch-testing-menu.png 219w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20220725165748\/batch-testing-menu-143x300.png 143w\" sizes=\"auto, (max-width: 219px) 100vw, 219px\" \/><\/a><\/p>\n<p><span style=\"font-weight: 400;\">To run a Test Suite, follow the steps below:<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Click the desired test suite name in the<\/span><b> Batch Testing<\/b><span style=\"font-weight: 400;\"> window.\u00a0<\/span><br \/>\n<span style=\"font-weight: 400;\">Note that <\/span><b>Developer Defined Utterances<\/b><span style=\"font-weight: 400;\"> and <\/span><b>Successful User Utterances<\/b><span style=\"font-weight: 400;\"> are default test suites to validate user utterances through Batch Testing.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Select <\/span><b>In Development<\/b><span style=\"font-weight: 400;\"> to run batch tests on test suites for only<\/span> the<b><i> in-development <\/i><\/b><span style=\"font-weight: 400;\">version of the Virtual Assistant(s). Alternatively, select <\/span><b>Published<\/b><span style=\"font-weight: 400;\"> to run batch tests on test suites for only<\/span> the published <span style=\"font-weight: 400;\">version of the Virtual Assistant(s).<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Click <\/span><b>Run Test Suite<\/b><span style=\"font-weight: 400;\"> to <\/span><span style=\"font-weight: 400;\">initiate the batch test execution.<\/span><br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230902153831\/run-test-suite-in-dev.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-59010\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230902153831\/run-test-suite-in-dev.png\" alt=\"\" width=\"650\" height=\"400\" \/><\/a><\/li>\n<\/ol>\n<h3><span class=\"ez-toc-section\" id=\"Execution_of_Newly_Created_Batch_Test_Suites\"><span style=\"font-weight: 400;\">Execution of Newly Created <\/span><span style=\"font-weight: 400;\">Batch Test Suites<\/span><\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Once you create a new test batch test suite, by default, the system automatically initiates test run execution for both <\/span><b><i>In-development <\/i><\/b><span style=\"font-weight: 400;\">and <\/span><b><i>Published<\/i><\/b><span style=\"font-weight: 400;\"> VA versions. <\/span><span style=\"font-weight: 400;\">This makes it convenient to assess test results while the VA is in development and once a VA is published following a production release.<\/span><br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230902154441\/batch-testing-run-publish.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-59013\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230902154441\/batch-testing-run-publish.png\" alt=\"\" width=\"650\" height=\"350\" \/><\/a><\/p>\n<div>\n<h3><span class=\"ez-toc-section\" id=\"Add_Notes_and_Comments_to_Batch_Testing\">Add Notes and Comments to Batch Testing<\/span><\/h3>\n<p>You can add notes or reasons to record the purpose or changes made for each test run when executing a batch test.<\/p>\n<p><strong>Steps to Add and Update a Batch Test Note<\/strong><\/p>\n<ol>\n<li>Click <strong>Run Test Suite <\/strong>to start batch test execution.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106084158\/run-test-suite.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60977\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106084158\/run-test-suite.png\" alt=\"\" width=\"600\" height=\"300\" \/><\/a><\/li>\n<li>When a batch test is running or completed, click the <strong>Notes<\/strong> icon for the required test run.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106090310\/notes-icon-batch-test.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60986\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106090310\/notes-icon-batch-test.png\" alt=\"\" width=\"600\" height=\"350\" \/><\/a><\/li>\n<li>Type your comment in the text window.<br \/>\n<b><\/b><\/li>\n<\/ol>\n<p><b>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0Add a New Note<\/b><\/p>\n<ul>\n<li>Type your comments in the <b>Add Notes<\/b> pop-up window and click <b>Save<\/b>.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106084719\/add-notes-batch-test.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-60980\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106084719\/add-notes-batch-test.png\" alt=\"\" width=\"369\" height=\"200\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106084719\/add-notes-batch-test.png 583w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106084719\/add-notes-batch-test-300x163.png 300w\" sizes=\"auto, (max-width: 369px) 100vw, 369px\" \/><\/a><\/li>\n<\/ul>\n<p><b>\u00a0 \u00a0 \u00a0 \u00a0 Note<\/b>:\u00a0You can add up to 1024 characters.<\/p>\n<p><b>\u00a0 \u00a0 \u00a0 \u00a0 Update an Existing Note<\/b><\/p>\n<ul>\n<li>Modify the text in the <b>Notes<\/b> window and click <b>Update<\/b>.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106085341\/update-batch-test-note.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60983\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20240106085341\/update-batch-test-note.png\" alt=\"\" width=\"350\" height=\"280\" \/><\/a><\/li>\n<\/ul>\n<ol start=\"4\">\n<li>After the test run is complete, you can click the <b>Notes<\/b> icon to view the new or updated comment.<\/li>\n<\/ol>\n<h3><span class=\"ez-toc-section\" id=\"Cancel_an_Ongoing_Batch_Test_Execution\"><span lang=\"EN\">Cancel an Ongoing Batch Test Execution<\/span><\/span><\/h3>\n<p>You can cancel an ongoing batch test execution in either <strong><em>Published<\/em><\/strong> or <strong><em>In-Development<\/em><\/strong> mode. This feature allows you to make changes, intervene, or start another batch test suite execution without waiting for the ongoing test run to complete.<\/p>\n<p>Steps to Cancel an Ongoing Batch Test Execution:<\/p>\n<ol>\n<li>Navigate to <strong>Build<\/strong> &gt; <strong>Testing<\/strong> &gt; <strong>Batch Testing<\/strong> on the Bot Builder.<\/li>\n<li>When the required test suite is running, click the <strong>Cancel<\/strong> icon.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118052438\/batch-testing-cancel-icon.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60107\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118052438\/batch-testing-cancel-icon.png\" alt=\"\" width=\"800\" height=\"350\" \/><\/a><\/li>\n<li>Click <b>Confirm<\/b> in the confirmation dialog.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118052655\/confirm-cancel-batch-testing.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60110\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118052655\/confirm-cancel-batch-testing.png\" alt=\"\" width=\"350\" height=\"275\" \/><\/a><\/li>\n<\/ol>\n<p>The test execution is canceled, and a confirmation message is displayed.<\/p>\n<p><b>Important Considerations<\/b><\/p>\n<ul>\n<li>For a canceled batch test suite, you cannot download the test results CSV file.<\/li>\n<li>An orange <strong>Warning<\/strong> icon indicates that the test suite execution is canceled.<\/li>\n<li>A red <strong>Warning<\/strong> icon indicates a failed batch test due to an unexpected technical error.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118054533\/warning-icon-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60113\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118054533\/warning-icon-1.png\" alt=\"\" width=\"800\" height=\"350\" \/><\/a><\/li>\n<\/ul>\n<p><strong>Error Scenarios<\/strong><\/p>\n<p>You cannot cancel a batch test run in the following scenarios:<\/p>\n<ol>\n<li><strong>\u00a0<\/strong>When another batch test execution\/cancelation is in progress.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118055253\/batch-test-in-progress-error.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60116\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118055253\/batch-test-in-progress-error.png\" alt=\"\" width=\"800\" height=\"275\" \/><\/a><br \/>\n<strong>Solution<\/strong>: Wait for one test suite execution or cancellation to complete before initiating the process for another.<\/li>\n<li>When you try to cancel a batch test run in the <b><i>Published<\/i><\/b> mode without publishing the VA.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118055741\/bot-not-published-yet.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-60119\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20231118055741\/bot-not-published-yet.png\" alt=\"\" width=\"800\" height=\"250\" \/><\/a><br \/>\n<b>Solution<\/b>: First, train and publish the VA before canceling the batch test run in Published mode.<\/li>\n<\/ol>\n<h3><span class=\"ez-toc-section\" id=\"View_Test_Case_Details\"><span style=\"font-size: 28px;\">View Test Case Details<\/span><\/span><\/h3>\n<p>Running a test suite will display the results as described below.<\/p>\n<div class=\"alert alert-info\"><b>Note:<\/b> The test case with multiple intents is considered a test case for all the intents mentioned. For example, if a test case has Intent 1 and Intent 2 as expected intents, then Intent 1 and Intent 2 are shown as covered in the batch test summary.<\/div>\n<p>Each test run creates a test report record and displays a summary of the test result. The batch test result in the screenshot below includes the following information:<\/p>\n<ul>\n<li><b>Last Run Date &amp; Time<\/b> that displays the date and time of the latest test run.<\/li>\n<li><b>F1 Score <\/b>is the weighted average of Precision and Recall i.e. (2*precision*recall)\/(precision+recall).<\/li>\n<li><b>Precision <\/b>is the number of correctly classified utterances divided by the total number of utterances that got classified (correctly or incorrectly) to an existing task ie the ratio of true positives to all classified positives (sum of true and false positives) i.e. TP\/(TP+FP).<\/li>\n<li><b>Recall <\/b>is the number of correctly classified utterances divided by the total number of utterances that got classified correctly to any existing task or classified incorrectly as an absence of an existing task ie the ratio of correctly classified utterances to actual matching intents\/tasks (sum of true positives and false negatives) i.e. TP\/(TP+FN).<\/li>\n<li><b>Intent Success %<\/b> that displays the percentage of correct intent recognition that has resulted from the test.<\/li>\n<li><b>Entity Success %<\/b> that displays the percentage of correct entities recognition that has resulted from the test.<\/li>\n<li><strong>Version Type<\/strong>\u00a0identifies the version of the bot against which the test suite was run &#8211; development or published.<\/li>\n<li>There are three possible outcomes from each test run:\n<ol>\n<li>Success &#8211; when all records are present in the file are processed<\/li>\n<li>Success with a warning &#8211; when one or more records present in the suite are discarded from detection due to system error<\/li>\n<li>Failed &#8211; when there was a system error and the test could not be resumed post-recovery.<\/li>\n<\/ol>\n<p>Hovering over the warning\/error icon will display a message suggesting the reason.<br \/>\n<a ref=\"magnificPopup\" href=\"\/wp-content\/uploads\/sites\/43\/Batch-Test-Sample-Result-warn.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-16623\" src=\"\/wp-content\/uploads\/sites\/43\/Batch-Test-Sample-Result-warn.png\" alt=\"\" width=\"750\" height=\"255\" \/><\/a><\/li>\n<\/ul>\n<h3><span class=\"ez-toc-section\" id=\"Download_a_CSV_Report\">Download a CSV Report<\/span><\/h3>\n<p>To get a detailed analysis of the test run, click <b>the Download<\/b> icon to download the test report in CSV format. You have an option to delete the test results if needed. The top section of the report comprises the summary with the following fields:<\/p>\n<ul>\n<li><strong>Bot Name<\/strong><\/li>\n<li><strong>Report name<\/strong> of the test suite<\/li>\n<li><strong>Bot Language<\/strong> (post 7.3 release)<\/li>\n<li><strong>Run Type<\/strong> identifies the version of the bot against which the test suite was run &#8211; development or published.<\/li>\n<li><strong>Threshold Setting<\/strong> (post 7.3 release) detailing the NLP thresholds applied when running this test suite, this would be followed by the settings for each of the three NL engines with the following details:\n<ul>\n<li><strong>Mode<\/strong> &#8211; ml, faq, or, cs<\/li>\n<li><strong>minThreshold<\/strong><\/li>\n<li><strong>maxThreshold<\/strong><\/li>\n<li><strong>exactMatchThreshold<\/strong><\/li>\n<li><strong>isActive<\/strong><\/li>\n<li><strong>taskMatchTolerance<\/strong><\/li>\n<li><strong>wordCoverage<\/strong><\/li>\n<li><strong>suggestionsCount<\/strong><\/li>\n<li><strong>pathCoverage<\/strong><\/li>\n<\/ul>\n<\/li>\n<li><b>Last Tested:<\/b> Date of the latest test run for developer-defined utterances.<\/li>\n<li><b>Utterance Count: <\/b>Total number of utterances included in the test run.<\/li>\n<li><b>Success\/Failure Ratio:<\/b> Total number of successfully predicted utterances divided by the total count of utterances multiplied by 100.<\/li>\n<li><b>True Positive (TP):<\/b> Percentage of utterances that have correctly matched expected intent.In the case of Small Talk, TP is indicated when the list of expected and actual intents are the same.In the case of Traits, a TP state includes the traits matched over and above the expected matches.\n<div class=\"alert alert-info\"><b>Important<\/b>: When you tag multiple intents (dialog, FAQ, and small talk) to a test utterance, the test result appears as TP if the actual intents match the expected intents but also include ambiguous intents.<\/div>\n<\/li>\n<li><b>True Negative (TN):<\/b> Percentage of utterances that were not expected to match any intent and they did not match. Not applicable to Small Talk.<\/li>\n<li><b>False Positive (FP):<\/b> Percentage of utterances that have matched an unexpected intent.\u00a0In the case of Small Talk, it would be when the list of expected and actual intents are different.<\/li>\n<li><b>False Negative (FN):<\/b> Percentage of utterances that have not matched expected intent.\u00a0In the case of Small Talk, it would be when the list of expected Small Talk intent is blank and but the actual Small Talk is mapped to an intent.<\/li>\n<\/ul>\n<p><a ref=\"magnificPopup\" href=\"\/wp-content\/uploads\/sites\/43\/Batch-Test-Sample-Result-2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-23825\" src=\"\/wp-content\/uploads\/sites\/43\/Batch-Test-Sample-Result-2.png\" alt=\"\" width=\"1303\" height=\"469\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20221201094601\/Batch-Test-Sample-Result-2.png 1303w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20221201094601\/Batch-Test-Sample-Result-2-300x108.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20221201094601\/Batch-Test-Sample-Result-2-1024x369.png 1024w\" sizes=\"auto, (max-width: 1303px) 100vw, 1303px\" \/><\/a><br \/>\nThe report also provides detailed information on each of the test utterances and the corresponding results.<\/p>\n<ul>\n<li><b>Utterances &#8211; <\/b>Utterances used in the corresponding test suite.<\/li>\n<li><b>Expected Intent <\/b>&#8211; The intent expected to match for a given utterance, will include trait where applicable with trait prefix<\/li>\n<li><b>Matched Intent<\/b> &#8211; The intent that is matched for an utterance during the batch test. This will include matched traits with trait prefix (post 7.3 release). This will include matched Small Talk intents (post 8.0 release).<\/li>\n<li><b>Parent Intent<\/b> &#8211; The parent intent considered for matching an utterance against an intent.<\/li>\n<li><b>Task State<\/b> &#8211; The status of the intent or task against which the intent is identified. Possible values include Configured or Published<\/li>\n<li><b>Result Type <\/b>&#8211; Result categorized as True Positive or True Negative or False Positive or False Negative<\/li>\n<li><strong>Entity Name<\/strong> &#8211; The name of the entity detected from the utterance.<\/li>\n<li><b>Expected EntityValue<\/b> &#8211; The entity value expected to be determined during the batch test.<\/li>\n<li><b>Matched EntityValue<\/b> &#8211; The entity value identified from an utterance.<\/li>\n<li><b>Entity Result<\/b> &#8211; Result categorized as True or False to indicate whether the expected entity value is the same as the actual entity value.<\/li>\n<li><strong>Expected Entity Order<\/strong> &#8211; entity values from the input file<\/li>\n<li><strong>Actual Entity Order<\/strong> &#8211;\n<ul>\n<li>if the order for all expected entities is provided, then the same is included in this column<\/li>\n<li>if no order is provided, the system determined order will be included in the column<\/li>\n<li>If an order is provided for some entities, then a combination of user-defined order and system-defined order will be included<\/li>\n<\/ul>\n<\/li>\n<li><strong>Matched Intent&#8217;s Score<\/strong> &#8211; For False Positives and False Negatives, the confidence scores from FM, ML, and\/or KG engines are displayed for the matched intent from the utterance. Note that the scores are given only if the engine detects the intent, which means that you may not see the scores from all three engines at all times.<\/li>\n<li><strong>Expected Intent&#8217;s Score<\/strong> &#8211; For False Positives, the confidence scores for the intent expected to match for the given utterance is given. Again the score will be given by the engines detecting the intent.<\/li>\n<\/ul>\n<div class=\"alert alert-info\"><b>Tip: <\/b>For any of the batch tests, if results indicate that your bot is unable to recognize the correct intents, you can work on improving its performance by adding or modifying utterances to the Machine Learning model.<\/div>\n<div><\/div>\n<div>\n<h3><span class=\"ez-toc-section\" id=\"Batch_Test_Results_for_Universal_Bots\"><span lang=\"EN\">Batch Test Results for Universal Bots<\/span><\/span><\/h3>\n<div>\n<p>The XO Platform now displays Batch Test Results for Universal bots for each test run in the summary view for <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/analyzing-your-bot\/virtual-assistants-health-and-monitoring\/#NLP-_FAQs_Summary\">FAQs<\/a>, <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/analyzing-your-bot\/virtual-assistants-health-and-monitoring\/#Dialog_Intent_Summary\">Dialog Intents<\/a>, and <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/analyzing-your-bot\/virtual-assistants-health-and-monitoring\/#NLP-_Small_Talk_Summary\">Small Talk<\/a> during Batch Testing. The actionable insights help understand the test coverage and NLP performance for each intent type in the Universal Bot. <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/analyzing-your-bot\/virtual-assistants-health-and-monitoring\/#NLP_Batch_Test_Results_Summary_for_Universal_Bots\" target=\"_blank\" rel=\"noopener noreferrer\">Learn more<\/a>.<\/p>\n<\/div>\n<\/div>\n<h3><span class=\"ez-toc-section\" id=\"View_Individual_Test_Run_Summary\">View Individual Test Run Summary<\/span><\/h3>\n<p>You can view the individual Batch Test Run summary of a test suite by clicking the <b>View Summary<\/b> icon of a test as shown below.<\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230629090751\/view-summary.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-57635\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v10-3\/wp-content\/uploads\/sites\/43\/20230629090751\/view-summary.png\" alt=\"\" width=\"700\" height=\"408\" \/><\/a><\/p>\n<p>This displays the relevant NLP and Flow Health metrics on the <b>Health and Monitoring<\/b> Dashboard for intents mapped to Dialogs, FAQs, and Traits. <a href=\"https:\/\/developer.kore.ai\/v10-3\/docs\/bots\/analyzing-your-bot\/virtual-assistants-health-and-monitoring\/#Test_Coverage\" target=\"_blank\" rel=\"noopener noreferrer\">Learn More<\/a>.<\/p>\n<div><\/div>\n<\/div>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n<div class=\"w-separator size_small with_line width_default thick_1 style_solid color_border align_center\"><div class=\"w-separator-h\"><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/section>\n","protected":false},"excerpt":{"rendered":"Once you have built and trained your bot, the most important question that arises is how good is your bot\u2019s learning model? So, evaluating your bot\u2019s performance is important to delineate how good your bot understands the user utterances. The Batch Testing feature helps you discern the ability of your bot to correctly recognize the...","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[46],"tags":[],"class_list":["post-18020","post","type-post","status-publish","format-standard","hentry","category-batch-testing"],"_links":{"self":[{"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/posts\/18020","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/comments?post=18020"}],"version-history":[{"count":120,"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/posts\/18020\/revisions"}],"predecessor-version":[{"id":61033,"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/posts\/18020\/revisions\/61033"}],"wp:attachment":[{"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/media?parent=18020"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/categories?post=18020"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/developer.kore.ai\/v10-3\/wp-json\/wp\/v2\/tags?post=18020"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}