Knowledge Graph Training

Training your Bot is not restricted to Machine Learning and Fundamental Meaning engines. You need to train the Knowledge Graph engine, too.

Knowledge Graph engine responds to users’ intents by identifying the appropriate questions from the Knowledge Graph.

Knowledge Graph

From the Knowledge Graph, follow these steps to build and train the corresponding Knowledge Graph:

1. Identify terms by grouping the unique words in each FAQ question. Build a hierarchy based on all such unique words.
2. Ensure that each node has not more than 25 questions.
3. Associate traits with terms to enable filtering FAQs from multiple identified results.
4. Define synonyms for each term/node in the hierarchy. Ensure that all the different ways to call the term are defined.
5. Depending on the importance of each term in a path, mark them as either mandatory or regular.
6. Define alternative questions for each FAQ to ensure better coverage.
7. Manage Context for accurate response.
8. Stop Words can be used to filter unwanted utterances.

Training and testing your Knowledge Graph is crucial to building an efficient Knowledge Graph.

Before training, you can improve performance by using tags, synonyms, and traits.

Term Type

(before ver 7.2, this was referred to as Term Usage under Manage Usage)

Designate the terms and tags in Knowledge Graph as Default, or Mandatory, or Organizer depending on their importance in qualifying matching paths:

• Default: Default terms do not have any particular considerations in shortlisting qualified paths.
• Mandatory: When you mark a term as Mandatory, all paths associated with the term are shortlisted for ranking only if the user’s utterance includes the mandatory term or its synonyms.
• Organizer: Term can be marked as being a part of the Knowledge Graph only for organizing questions (this option is available only for terms not tags).

Tags

When you type a question in the User Says field, the Knowledge Graph suggests some tags that you can add to the graph based on the text. To include a suggested term to the path, select the tag from the drop-down list that appears when the cursor is in the Add Term field. You can also add custom tags by typing them in the add term field and hitting the enter key.

Once you add a tag, it will be visible below the question like a tag everywhere the question appears. Tags work exactly like terms but are not displayed in the Knowledge Graph to avoid clutter. You can add synonyms and traits to tags as you do to terms.

Synonyms

You can add multiple synonyms for each term in your Knowledge Graph, making the path discoverable for varied user utterances. You can add synonyms for a term from the Settings window.

When you add a synonym for a term in the Knowledge Graph, you can add them as local (Path Level) or global (Knowledge Graph) synonyms.

Local synonyms apply to the term only in that particular path, whereas global synonyms apply to the term even if it appears on any other path in the hierarchy.

To add synonyms for a term:

1. On the top left side of the bot’s Knowledge Graph, hover over the node/term for which you want to add synonyms.
2. Click the gear icon to open the Settings window.
3. To add synonyms,  do the following:
   • To add local synonyms, type them in the box under Path Level Synonyms.
     
   • To add Global synonyms, click Edit or Add New under Knowledge Graph Synonyms and enter them. These Graph Synonyms can also be accessed from the Manage Synonyms option under the more options icon on the top right corner of the Knowledge Graph page.
     
     Note: Press Enter after typing each synonym in the Synonyms box. If you type multiple synonyms without pressing enter after each synonym, all the synonyms are considered as a single entity, even if they are separated by spaces.
   • Post release 7.2, you can use Bot Synonyms in the identification of KG terms. This option can be enabled either from the Threshold and Configurations or from the Manage Synonyms option under the more options icon on the top right corner of the Knowledge Graph page.
     
     Once enabled, the bot level synonyms that match with KG terms (or tags) will be automatically displayed under the Bot Synonyms heading in Synonyms section and will be used by the KG engine. The Bot Synonyms will be used, similar to that of KG graph level synonyms, for path qualification and for question matching. When a node matches both with a bot synonym and a bot concept, the bot concept will take priority.
     
4. To add synonyms for a child node, enter them in the Synonyms box next to the Child Terms listed at the bottom of the settings window.

To add synonyms for a tag:

1. On the top left side of the Knowledge Graph, click the term/node to which you have added the question.
2. From the list of questions on the Questions panel, hover over the question.
3. Click the edit icon and in the Edit Q&A window, double-click the tag. (You can edit custom tags alone not the default ones)
4. On the Tag Settings window that opens, type each synonym and press Enter to add it.

Traits

NOTE: Traits replace Classes from before ver 6.4.

You can create traits with common user utterances and then add them to the relevant terms – nodes and tags, in your Knowledge Graph. To know more about Traits, click here.

To create a trait:

Traits are common across the Bot Builder if you have created Traits from Natural Language section they are available for use here.

1. On the top right side of the Knowledge Graph window, click the more options icon and then select Manage Traits.
   
2. In the Manage Traits pop-up window, click New Trait.
3. In the Trait Type and Trait Name field, enter a relevant name for the trait, for example, Issues.
4. In the Utterances box, enter all the utterances that you want to include in the trait. Examples of the Issues trait: it is not working, not working, is not working, and I cannot see.
   
5. Click Save.

After you create a trait, you can assign it to multiple nodes and tags in the Knowledge Graph.

To add a trait to a node/term:

1. On the top left side of your Knowledge Graph, hover over the terms to which you want to add the trait.
2. Click the gear icon to open the Settings window.
3. In the Trait drop-down list, select the name of a trait and then click Save.

1. On the top left side of your Knowledge Graph, click the term to which you have added the question.
2. From the list of questions on the Questions panel, hover over the question.
3. Click the edit icon and in the Edit Q&A window, double-click the term. (You can edit custom terms alone not the default ones)
4. From the Tag Settings window, in the Traits drop-down list, select the name of a trait and then click Save.

Context

You can Manage Context for the terms and tags by setting:

• the Intent Precondition – the context that should be present as a qualifier for this node or tag
• Context Output – the context that should be populated to signify the execution of this task

Post the release of ver8.0 of the platform, context can be enabled for Organizer nodes as well. Enabling the Manage Context option would allow you to set context precondition and context output mentioned above. NOTE enabling the manage context option will not emit the term/node name by default.

See here for more on Context Management.

Stop Words

Stop words present in the user utterance are discarded for scoring even if the stop word is used to define a node (or node synonyms).

Knowledge Graph has a language-specific predefined set of stop words. This list can be customized to suit your requirements.

To edit the stop words list:

1. From the Knowledge Graph page, click on the more options icon and select Manage Stop Words
   
2. From the Manage Stop Words window, delete or add stop words.

Train & Test

After you complete creating/editing the Knowledge Graph, click the Train button on the top right-hand side of your Knowledge Graph window. When you do so, all the paths, synonyms, and question-answer sets are sent to the Graph DB engine.

The training would fail if any single node has more than 100 questions. This limit was introduced in ver7.3 to make Knowledge Graph more efficient by improving the response times. In such failure cases, you can Download Errors CSV file which lists the path with more than 100 questions. You can use this file to rectify your Knowledge Graph.

Test the Knowledge Graph

When you complete creating the Knowledge Graph and training it, we recommend you to interact with the bot and ask questions connected to the Knowledge Graph. Test the bot responses by using a variety of utterances so that you can identify missing terms, questions, alternative questions, synonyms, and traits.

FAQ Detection Steps

The following FAQ Detection steps give you an overview of the process in which the Knowledge Graph(KG) engine shortlists the questions from a Knowledge Graph:

1. Extract Nodes: The KG engine processes the user utterance to extract the term (nodes) present in the Knowledge Graph. It also takes into consideration the synonyms, traits, and tags associated with the terms.
2. Query Graph: The KG engine fetches all the paths that consist of the extracted nodes.
3. Shortlist Paths: All the paths consisting of 50% or more matching terms with the user utterance are shortlisted for further processing. For example, the engine shortlists a path with four nodes such as Personal Banking → Joint Account → Add → Account Holder if at least two of these terms occur in the user utterance.
   Note: Path coverage computation doesn’t consider the root node.
4. Filter with Traits: If you define any traits in the Knowledge Graph, paths shortlisted in the above step are further filtered based on the confidence score of a classification algorithm in user utterance.
5. Send to Ranker: The KG engine then sends the shortlisted paths to the Ranker Program.
6. Score based on Cosine Similarity: Ranker makes use of user-defined synonyms, lemma forms of word, n-grams, stop words, to compute the cosine similarity between user utterance and the shortlisted questions. Paths are ranked in non-increasing order of cosine similarity score.
7. Qualify Matches: The ranker then qualifies the paths as follows:
   • Paths with score >= upper_threshold are qualified as an answer (definitive match).
   • Paths with lower_threshold < score < upper_threshold are marked as suggestion (probable match).
   • Paths with score < lower_threshold are ignored.

Threshold & Configurations

To train and improve the performance, Threshold and Configurations can be specified for all three NLP engines – FM, KG, and ML. You can access these settings from Natural Language > Training > Thresholds & Configurations.

NOTE: If your Bot is multilingual, you can set the Thresholds differently for different languages. If not set, the Default Settings will be used for all languages. This feature is available from ver 7.0 onwards.

The settings for the Knowledge Graph engine are discussed in detail in the following sections.

Navigating to Threshold and Configurations

1. Open the bot for which you want to configure Knowledge Graph settings.
2. Hover over the side navigation panel and then click Natural Language -> Training.
3. Click the Thresholds & Configurations tab.
4. Below is a detailed discussion about the Knowledge Graph section on this page.
   

Auto Correction will spell correct the words in the user input to the closest matching word from the Bot’s Knowledge Graph domain dictionary. Knowledge Graph domain dictionary comprises of the words extracted from Knowledge Graph’s questions, alternate questions, nodes, and synonyms.

Bot Synonyms (introduced in ver 7.2) will enable the Bots platform to use the Bot Synonyms in Knowledge Graph as well. Inclusion of Bot Synonyms for intent detection by the KG engine requires training. Click ‘Proceed’ when prompted to enable this setting and initiate training.

Lemmatization using Parts of Speech (introduced in ver 7.3) will enable the use of parts of speech associated with the words in the utterance to lemmatize. (see below for more)

Path Coverage can be used to define the minimum percentage of terms in the user’s utterance to be present in a path to qualify it for further scoring. The default setting is 50% i.e. at least half of the terms in the user utterance should match the node names & terms.

Minimum and Definitive Level for Knowledge Graph Intent allows you to set the confidence levels for a Knowledge Graph intent. You can view and adjust the confidence level percentages for the graph in one of three ranges as:

• Definitive range – Matches in this range (green area) are picked and any other probable matches are discarded, default set to 93-100%.
• Probable range – Matches in this range (dark gray area) are considered for rescoring and ranking, by default set to 80-93%
• Low confidence range – If no other intents have matched, low confidence matches (orange area) are presented to end-user for intent confirmation, by default set to 60-80%
• Not matching an intent – The light gray area represents the knowledge graph intent NLP interpreter confidence levels as too low to match the knowledge graph intent, default set to 60%.

KG Suggestions Count: Define the maximum number of KG / FAQ suggestions (up to 5) to be presented when a definite KG intent match is not available. Default set to 3.

Proximity of Suggested Matches: Define the maximum difference (up to 50%) to be allowed between top-scoring and immediate next suggested questions to consider them as equally important. Default set to 5%. This applies to the matches in the Probable range.

Manage Long Responses when the response size exceeds channel-specific limitations. You can choose to truncate the response or display the full response with a read more link. ‘Read More’ link will be included at the end of the message. On selecting this link, the full response will be opened as an answer in the browser. The URL to open the long response in a web browser is set by default by the platform. But you can provide a custom URL, too.

Search in Answer for the qualifying FAQs (see below for more)

Qualify Contextual Paths in the Knowledge Graph using the context tags available in the context. Enabling this option will ensure that the paths will be shortlisted using terms or tags from the context. These tags can come from previous matched path or intent or custom defined tags.

In ver8.0 of the platform, some advanced configurations have been introduced. Refer here for more details.

Search in Answer

This feature will enable identifying FAQs by searching the user input against the answer section, instead of only matching with questions. This will be a fallback mechanism only i.e. search in the answer section will be done only if no FAQs are identified from questions.

Note: This feature is not supported in all languages, see here for details.

When the ‘Search in Answer‘ flag is enabled, the Knowledge Graph engine will consider the answer text for identifying the intents also.

Once this option is enabled, you can specify whether to Inform the end user that the answer is a probable answer. If selected a Standard Message to the effect will be displayed, which can be customized using the Manage Response link. Know more.

There are three ways in which you can render the response:

1. Show complete response: Full response will be sent as the answer to the user.
2. Show only the relevant paragraph: Only the relevant paragraph from which the question was identified will be sent as the response.
3. Show only the relevant paragraph with ‘Read More’ link: Only the relevant paragraph from which the question was identified will be sent as the response.
   An additional ‘Read More’ link will be included at the end of the message. On selecting this link, the full response will be opened as an answer in the browser. The URL to open the long response in a web browser is set by default by the platform. But you can provide a custom URL (see below for details).

Custom URL Configuration:

By default, the URL to open the long response in the web browser is set by the platform. You have an option to provide a custom URL for rendering the FAQ answers.

The platform will call the provided URL with details of the relevant message template (template id) and other necessary information.

The following API gives the full information of the FAQ:

URL:
https://{{host-name}}/api/1.1/public/users/{{userId}}/faqs/resolvedResponse/{{respId}}

Method: get

Headers: {auth : JWT}

Sample Response:
{
"response": "You can contact our Branch officials wherein you have submitted your documents.If the documents are in order, the account will be opened within 2 working days.",
"primaryQuestion": "How to check the status of my account opening?"
}

Lemmatization

Lemmatization in linguistics is the process of grouping together the inflected forms of a word so they can be analyzed as a single item, identified by the word’s lemma, or dictionary form. Using Parts of Speech information from the user utterance in the process of lemmatization can improve identifying a more accurate FAQ.

Following are some examples of the phrases as recognized by the KG engine with and without using parts of speech: