{"id":43263,"date":"2022-07-25T08:44:04","date_gmt":"2022-07-25T07:44:04","guid":{"rendered":"https:\/\/developer.kore.ai\/v9-3\/?p=43263"},"modified":"2023-01-16T15:39:21","modified_gmt":"2023-01-16T15:39:21","slug":"training-validations","status":"publish","type":"post","link":"https:\/\/developer.kore.ai\/v9-3\/docs\/bots\/nlp\/training-validations\/","title":{"rendered":"Training Validations"},"content":{"rendered":"<section class=\"l-section wpb_row height_auto\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row type_default valign_top\"><div class=\"vc_col-sm-12 wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\r\n\t<div class=\"wpb_text_column \">\r\n\t\t<div class=\"wpb_wrapper\">\r\n\t\t\t<p>NLP models play a significant role in providing natural conversational experiences for your customers and employees. Improving the accuracy of the NLP models is a continuous journey and requires fine-tuning, as you add new use cases to your virtual assistant. The Kore.ai XO Platform proactively validates the NLP training provided to the virtual assistants and provides recommendations to improve the model. This article explains the available validations, how to view these validations, and how to validate the NLU Model.<\/p>\n<h2 id=\"goal-driven\"><span class=\"ez-toc-section\" id=\"Goal-Driven_Training_Validations\">Goal-Driven Training Validations<\/span><\/h2>\n<p><span style=\"font-weight: 400\">The ML engine enables you to identify issues proactively in the training phase itself with the following set of recommendations:\u00a0<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><b>Untrained Intents<\/b><span style=\"font-weight: 400\"> &#8211; notifies about intents that are not trained with any utterances so that you can add the required training.\u00a0<\/span><\/li>\n<p \/>\n<li style=\"font-weight: 400\"><b>Inadequate training utterances<\/b><span style=\"font-weight: 400\"> &#8211; notifies the intents with insufficient training utterances so that you can add more utterances to them.\u00a0<\/span><\/li>\n<p \/>\n<li style=\"font-weight: 400\"><b>Utterance does not qualify any intent (false negative)<\/b><span style=\"font-weight: 400\"> &#8211; notifies about an utterance for which the NLP model cannot predict any intent. For example, an utterance added to Intent A is expected to predict Intent A. Whereas in some cases the model won\u2019t be able to predict neither the trained Intent A nor any other Intents within the model. Proactively identifying such cases helps you rectify the utterance and enhance the model for prediction.\u00a0<\/span><\/li>\n<p \/>\n<li style=\"font-weight: 400\"><b>Utterance predicts wrong intent (false positive)<\/b><span style=\"font-weight: 400\"> &#8211;<\/span><span style=\"font-weight: 400\"> Identifies utterances that predict intents other than the trained intent. For example, when you add an utterance similar to utterances from another intent, the model could predict a different intent rather than the intent it is trained to. Knowing this would help you to rectify the utterance and improve the model prediction. <\/span><\/li>\n<p \/>\n<li style=\"font-weight: 400\"><b>Utterance predicts intent with low confidence <\/b><span>&#8211; notifies about the utterances that have low confidence scores. With this recommendation, you can identify and fix such utterances to improve the confidence score during the virtual assistant creation phase.<\/span><\/li>\n<p \/>\n<li style=\"font-weight: 400\"><b>Incorrect Patterns <\/b><span>&#8211;  notifies about the patterns that do not follow the right syntax along with the error. You can resolve such incorrect patterns to improve intent identification.<\/span><\/li>\n<p \/>\n<li style=\"font-weight: 400\"><b>Wrong Entity Annotations <\/b><span>&#8211;  notifies wrongly annotated entities. For example, in the utterance <em>\u2018I want to travel to Hyderabad on Sunday 2pm\u2019,<\/em>. the<em> Travel Date<\/em> (Date type) entity is annotated with value <em>\u20182PM\u2019 (Time value)<\/em>. The platform checks for such wrong annotations and notifies the issue against the utterance, which helps to re-annotate the entity with the right values and improve entity recognition.<\/span><\/li>\n<p \/>\n<li style=\"font-weight: 400\"><b>Short Utterance <\/b><span>&#8211;  notifies about the utterances whose word count is lesser than or equal to two. It helps you to follow best practices for the length of utterances, which depicts an actual end-user query and further improves the model&#8217;s accuracy.<\/span><\/li>\n<\/ul>\n<h2><span class=\"ez-toc-section\" id=\"How_to_View_NLU_Training_Validations\"><span style=\"font-weight: 400\">How to View NLU Training Validations<\/span><\/span><\/h2>\n<ol>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">On the virtual assistant\u2019s <\/span><b>Build <\/b><span style=\"font-weight: 400\">menu, click <\/span><b>Natural Language -&gt;<\/b> <b>Training.<\/b><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">In the <\/span><b>Intents <\/b><span style=\"font-weight: 400\">tab, you can see the set of recommendations for the Intents and ML utterances.<br \/><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093245\/NLU_recommendations.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093245\/NLU_recommendations.png\" alt=\"\" width=\"600\" height=\"400\" class=\"alignnone size-medium wp-image-40424\" \/><\/span><span style=\"font-weight: 400\"><\/a><br \/><\/span><b>Note<\/b><span style=\"font-weight: 400\">: The errors and warnings in this screen are examples. The ML validations vary based on the error or waning recommendation as explained in the Goal-Based NLU Training Validations section above.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Hover over the validation options and view the following recommendations:<\/span>\n<ul>\n<li>Hover on the <\/span> <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093517\/error_icon.png\" alt=\"\" width=\"12\" height=\"12\" \/> <b>Error <\/b><span style=\"font-weight: 400\">icon<\/span> <span style=\"font-weight: 400\">to view the recommendations to resolve the error. <a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093426\/error.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093426\/error.png\" alt=\"\" width=\"800\" height=\"400\" class=\"alignnone size-medium wp-image-40429\" \/><\/a><\/span><\/br><\/li>\n<p><b>Note<\/b>: An Error is displayed when the intent has a definite problem that impacts the virtual assistant\u2019s accuracy or intent score. Errors are high severity problems.<\/li>\n<li>Hover on the <\/span> <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093604\/warning_icon.png\" alt=\"\" width=\"12\" height=\"12\" \/> <b>Warning <\/b><span style=\"font-weight: 400\">icon and follow the instructions in the warning to enhance the training for ML utterances.<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093735\/warning_recommendation.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220409093735\/warning_recommendation.png\" alt=\"\" width=\"800\" height=\"400\" class=\"alignnone size-medium wp-image-40438\" \/><\/a><\/span><\/br><b><\/li>\n<\/ul>\n<p>Note<\/b>:  A warning is displayed when the issue impact the VA\u2019s accuracy and it can be resolved. Warnings are less severe problems when compared to errors.<\/li>\n<li>Once you click on the Intent with error or warning, hover over the <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220411110705\/bulb_icon.png\" alt=\"\" width=\"12\" height=\"12\" \/> <b>Bulb<\/b> icon to view the summary of error or warning messages as illustrated below:<br \/><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220411110802\/bulb_details.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220411110802\/bulb_details.png\" alt=\"\" width=\"600\" height=\"400\" class=\"alignnone size-medium wp-image-40514\" \/><\/a><\/li>\n<\/ol>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n<\/div><\/div><\/div><\/div><\/div><\/section><section class=\"l-section wpb_row height_auto\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row type_default valign_top\"><div class=\"vc_col-sm-12 wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\r\n\t<div class=\"wpb_text_column \">\r\n\t\t<div class=\"wpb_wrapper\">\r\n\t\t\t<h2><span class=\"ez-toc-section\" id=\"How_to_Use_the_NLU_Validate_Model\"><span style=\"font-weight: 400\">How to Use the NLU Validate Model<\/span><\/span><\/h2>\n<p>The NLU Validate Model provides the options to view  the training module&#8217;s <strong>recommendations summary<\/strong>, <strong>refresh the recommendations<\/strong>, and view Confusions Matrix, and K-fold Cross Validation reports.<\/p>\n<p>The Validate Model helps you to follow best practices and develop accurate Virtual Assistants quickly.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Recommendations_Summary\">Recommendations Summary<\/span><\/h3>\n<p>In the <strong>Natural Language \u2192 Training<\/strong> page, click the <strong>Validate Model<\/strong> to see the summary of recommendations.<\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725091925\/Val-Model.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725091925\/Val-Model.png\" alt=\"\" width=\"540\" height=\"333\" class=\"alignnone size-large wp-image-43292\" \/><\/a><\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725092214\/Recomm.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725092214\/Recomm.png\" alt=\"\" width=\"370\" height=\"293\" class=\"alignnone size-full wp-image-43296\" \/><\/a><\/p>\n<div class=\"alert alert-info\"><strong>Note:<\/strong> The recommendations are listed and categorized in the preceding screenshot. Under the <strong>All<\/strong> tab, all errors and warnings are listed. Under the <strong>Error <\/strong>and <strong>Warning <\/strong>tabs, errors and warnings are listed, respectively.<\/div>\n<p>In the recommendation, <i>5 intents have Patterns with invalid syntax<\/i>, 5 is the count of the intents with the issue &#8211; patterns with invalid syntax. For any recommendation, the intent count is displayed first, followed by the type of the issue.<\/p>\n<p>If the training model is updated with all the modifications, the platform automatically triggers a background task to refresh recommendations without explicitly informing the users.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Refreshing_Recommendations\">Refreshing Recommendations<\/span><\/h3>\n<p>In the recommendation summary, you can refresh recommendations as explained in the previous section in some of the scenarios like:<\/p>\n<li><strong>Modify training data:<\/strong> When the training data is modified, you can check for new recommendations.<\/li>\n<li><strong>Implement recommendations:<\/strong> When a recommendation is implemented, you can verify if it is resolved.<\/li>\n<li><strong>Add new training data:<\/strong> When new intents or utterances are added, you can check for new recommendations.<\/li>\n<p>When you click the <strong>Refresh<\/strong> icon, a timestamp of the last refresh is displayed along with the refreshed recommendations list. The timestamp is updated with every refresh.<\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725094425\/refresh1.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725094425\/refresh1.png\" alt=\"\" width=\"425\" height=\"308\" class=\"alignnone size-full wp-image-43302\" \/>&#8220;<\/a><\/p>\n<p>If there are untrained utterances in the training model, the platform provides you with the following options:<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Train_and_Regenerate\">Train and Regenerate<\/span><\/h3>\n<p>This option allows you to train the model with untrained utterances, and then triggers a background task to generate recommendations from the latest model.<\/p>\n<p>The following steps explain the <strong>Train &amp; Generate <\/strong>usage with an example.<\/p>\n<ol>\n<li>Click any intent and add a new utterance.<\/li>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220726115138\/Train1-1.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220726115138\/Train1-1.png\" alt=\"\" width=\"640\" height=\"236\" class=\"alignnone size-large wp-image-43573\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220726115138\/Train1-1.png 1482w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220726115138\/Train1-1-300x111.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220726115138\/Train1-1-1024x378.png 1024w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<li>Once the utterance is added, go to the main page and click the <b>Refresh<\/b> icon to refresh the recommendations. The following pop-up is displayed when there are untrained utterances.<\/li>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095047\/Train3.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095047\/Train3.png\" alt=\"\" width=\"561\" height=\"393\" class=\"alignnone size-full wp-image-43309\" \/><\/a><\/p>\n<li>Click the <b>Train &amp; Regenerate<\/b> button to train the utterances and regenerate the recommendations.<\/li>\n<li>A message about training initiation is displayed.<\/li>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095415\/TrainINit.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095415\/TrainINit.png\" alt=\"\" width=\"602\" height=\"123\" class=\"alignnone size-full wp-image-43313\" srcset=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095415\/TrainINit.png 602w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095415\/TrainINit-300x61.png 300w, https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095415\/TrainINit-600x123.png 600w\" sizes=\"auto, (max-width: 602px) 100vw, 602px\" \/><\/a><\/p>\n<li>Once training and regeneration of recommendations is completed, the status is displayed in the status docker.<\/li>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095517\/Status-Docker.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095517\/Status-Docker.png\" alt=\"\" width=\"449\" height=\"251\" class=\"alignnone size-full wp-image-43316\" \/><\/a><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Regenerate\">Regenerate<\/span><\/h3>\n<p>Click the <strong>Regenerate<\/strong> button in the displayed pop-up if you want to trigger a background task that generates recommendations from the current model.<\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095047\/Train3.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095047\/Train3.png\" alt=\"\" width=\"461\" height=\"293\" class=\"alignnone size-full wp-image-43309\" \/><\/a><\/p>\n<div class=\"alert alert-info\"><strong>Note:<\/strong> While performing the refresh, you cannot trigger the task again.<br \/>\nIf you click the <strong>Refresh<\/strong> icon when the model is being trained, recommendations from the latest Validate model are generated only once the training is completed.<\/div>\n<li>When you click the <strong>Refresh<\/strong> icon, the recommendations summary is refreshed and the count of recommendations is either increased, or decreased, or the recommendations are updated based on the latest results.<\/li>\n<\/ol>\n<p>For example, if new utterances are added to the intent, a new recommendation may get added to the summary list. Similarly, if a recommendation is implemented, the count of recommendations decreases. In a recommendation like <em>13 intents have very short utterances, if 2 intents are fixed, then the recommendation is updated to 11 intents have very short utterances.<\/em><\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095937\/Refresh.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725095937\/Refresh.png\" alt=\"\" width=\"487\" height=\"302\" class=\"alignnone size-full wp-image-43319\" \/><\/a><\/p>\n<h3><span class=\"ez-toc-section\" id=\"NLU_Validation_Options\">NLU Validation Options<\/span><\/h3>\n<p>Next to the <strong>Validate Model<\/strong>, there is a drop-down with options to select either the <strong>Confusion matrix<\/strong> or <strong>K-fold Cross Validations<\/strong>.<\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725100230\/Dropdown.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725100230\/Dropdown.png\" alt=\"\" width=\"482\" height=\"220\" class=\"alignnone size-full wp-image-43323\" \/><\/a><\/p>\n<p>You can also click the <strong>Go<\/strong> button to access Confusion Matrix and K-fold Cross Validation reports.<\/p>\n<p><a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725100330\/go.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725100330\/go.png\" alt=\"\" width=\"502\" height=\"247\" class=\"alignnone size-full wp-image-43326\" \/><\/a><\/p>\n<h4>Confusion Matrix<\/h4>\n<p>Confusion Matrix is useful in describing the performance of a classification model (or classifier) on a set of test data for which the true values are known. The graph generated by the confusion matrix presents an at-a-glance view of the performance of your trained utterances against the virtual assistant&#8217;s tasks. To learn more, see the Confusion Matrix section in <a href=\"https:\/\/developer.kore.ai\/v9-3\/docs\/bots\/nlp\/ml-model\/\">Model Validation<\/a>.<\/p>\n<p>The following screenshot shows the confusion matrix report.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725100449\/ConfMatrix.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725100449\/ConfMatrix.png\" alt=\"\" width=\"840\" height=\"327\" class=\"alignnone size-large wp-image-43329\" \/><\/a><\/p>\n<li>If no data is available in the Confusion Matrix, you click the <strong>Generate<\/strong> button to create a report.<\/li>\n<li>Whenever the Validate model is updated, you can click the <strong>Re-Run Model<\/strong> to generate the latest matrix. Once you rerun the model, the platform prompts you to either <a href=\"https:\/\/developer.kore.ai\/v9-3\/docs\/bots\/nlp\/training-validations\/#Train_and_Regenerate\">Train and Regenerate<\/a> the recommendations if the model has unsaved changes or <a href=\"https:\/\/developer.kore.ai\/v9-3\/docs\/bots\/nlp\/training-validations\/#Regenerate\">Regenerate <\/a>the recommendations if the model is up to date.<\/li>\n<li>Whenever you <a href=\"https:\/\/developer.kore.ai\/v9-3\/docs\/bots\/nlp\/training-validations\/#Train_and_Regenerate\">Train and Regenerate<\/a> the recommendations, the matrix will also be regenerated. Similarly, whenever you regenerate the matrix, the recommendations are updated too.<\/li>\n<h4>K-Fold Cross Validation<\/h4>\n<p>K-Fold Cross-Validation is a resampling procedure used to evaluate machine learning models on a limited data sample. The technique involves partitioning the data into subsets, training the data on a subset, and using the other subsets to evaluate the model\u2019s performance. To learn more, see the K-Fold Cross Validation section in <a href=\"https:\/\/developer.kore.ai\/v9-3\/docs\/bots\/nlp\/ml-model\/\">Model Validation<\/a>.<\/p>\n<p>The following screenshot shows the K-Fold Cross-Validation report.<br \/>\n<a ref=\"magnificPopup\" href=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725111525\/Kfold2.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/kore-wordpress.s3.us-east-2.amazonaws.com\/developer.kore.ai\/v9-3\/wp-content\/uploads\/sites\/29\/20220725111525\/Kfold2.png\" alt=\"\" width=\"640\" height=\"284\" class=\"alignnone size-large wp-image-43374\" \/><\/a><\/p>\n\r\n\t\t<\/div>\r\n\t<\/div>\r\n<div class=\"w-separator size_medium\"><\/div><\/div><\/div><\/div><\/div><\/div><\/section><section class=\"l-section wpb_row height_auto\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row type_default valign_top\"><div class=\"vc_col-sm-12 wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">[vc_raw_html]JTNDZGl2JTIwY2xhc3MlM0QlMjJ3LXBvc3QtZWxtJTIwcG9zdF9uYXZpZ2F0aW9uJTIwbGF5b3V0X3NpbXBsZSUyMGludl9mYWxzZSUyMiUzRSUwQSUzQ2ElMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtJTIwb3JkZXJfZmlyc3QlMjB0b19wcmV2JTIyJTIwaHJlZiUzRCUyMiUyRmRvY3MlMkZib3RzJTJGbmxwJTJGdHJhaXRzJTJGJTIyJTNFJTNDJTJGcCUzRSUwQSUzQ2RpdiUyMGNsYXNzJTNEJTIycG9zdF9uYXZpZ2F0aW9uLWl0ZW0tYXJyb3clMjIlM0UlM0MlMkZkaXYlM0UlMEElM0NkaXYlMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtLW1ldGElMjIlM0VQcmV2aW91cyUyMFBvc3QlM0MlMkZkaXYlM0UlMEElM0NkaXYlMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtLXRpdGxlJTIyJTNFJTNDc3BhbiUzRVRyYWl0cyUyMEVuZ2luZSUzQyUyRnNwYW4lM0UlM0MlMkZkaXYlM0UlMEElM0NwJTNFJTNDJTJGYSUzRSUzQ2JyJTIwJTJGJTNFJTBBJTNDYSUyMGNsYXNzJTNEJTIycG9zdF9uYXZpZ2F0aW9uLWl0ZW0lMjBvcmRlcl9zZWNvbmQlMjB0b19uZXh0JTIyJTIwaHJlZiUzRCUyMiUyRmRvY3MlMkZib3RzJTJGbmxwJTJGbmxwLWRldGVjdGlvbiUyRiUyMiUzRSUzQyUyRnAlM0UlMEElM0NkaXYlMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtLWFycm93JTIyJTNFJTNDJTJGZGl2JTNFJTBBJTNDZGl2JTIwY2xhc3MlM0QlMjJwb3N0X25hdmlnYXRpb24taXRlbS1tZXRhJTIyJTNFTmV4dCUyMFBvc3QlM0MlMkZkaXYlM0UlMEElM0NkaXYlMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtLXRpdGxlJTIyJTNFJTNDc3BhbiUzRVJhbmtpbmclMjBhbmQlMjBSZXNvbHZlciUzQyUyRnNwYW4lM0UlM0MlMkZkaXYlM0UlMEElM0NwJTNFJTNDJTJGYSUzRSUzQyUyRmRpdiUzRQ==[\/vc_raw_html]<\/div><\/div><\/div><\/div><\/div><\/section>\n","protected":false},"excerpt":{"rendered":"NLP models play a significant role in providing natural conversational experiences for your customers and employees. Improving the accuracy of the NLP models is a continuous journey and requires fine-tuning, as you add new use cases to your virtual assistant. The Kore.ai XO Platform proactively validates the NLP training provided to the virtual assistants and...","protected":false},"author":20,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-43263","post","type-post","status-publish","format-standard","hentry","category-nlp"],"_links":{"self":[{"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/posts\/43263","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/users\/20"}],"replies":[{"embeddable":true,"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/comments?post=43263"}],"version-history":[{"count":62,"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/posts\/43263\/revisions"}],"predecessor-version":[{"id":51276,"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/posts\/43263\/revisions\/51276"}],"wp:attachment":[{"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/media?parent=43263"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/categories?post=43263"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/developer.kore.ai\/v9-3\/wp-json\/wp\/v2\/tags?post=43263"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}