Fundamental Meaning

Fundamental Meaning is a computational linguistics approach that’s built on ChatScript. The model analyzes the structure of a user’s utterance to identify each word by meaning, position, conjugation, capitalization, plurality, and other factors.

In this section we will discuss in details the following topics to improve the FM engine:

• Managing Synonyms & Concepts more…
• Managing Patterns & Rules more…
• Setting Thresholds more…
• Negative Patterns more…

Managing Synonyms

Users utter the same request in varied ways.

For example, an intent like Create Lead could be expressed with alternatives such as “New Lead,” “Make a New Lead,” or “Generate a Lead.”

As a developer, you should limit the name of a task to only two or three words, yet accommodate the alternative ways in which it could be asked.

Synonyms can be defined for both the task name and a task field name. To optimize the NLP interpreter accuracy in recognizing the correct task and task field provided by the user, you can add synonyms for words used in the names of your tasks and task fields.

• Create – new, build, design, generate, instantiate, make, produce, and so forth.
• Lead – sales lead, customer lead, potential lead, qualified lead, and so forth.

Also consider misspellings, such as:

• Create – crate, creeate, etc.
• Lead – leed, led, and lede.

When you add synonyms for a word, those synonyms are also used for other bot tasks or task fields. In other words, synonyms for a word defined for a task are also applicable to the same word in another task name, for example, synonyms defined for the word create in the Create a New Lead task are also used for the Create Opportunity task.

For more information about best practices and tips for naming tasks and task fields, see the Natural Language Processing Guide.

Navigating to the Bot Synonyms

1. Open the bot for which you want to add synonyms.
2. Hover over the side navigation panel and then click Natural Language -> Training.
3. Click the Synonyms and Concepts tab.
4. On the Synonyms page, click one of the following tabs:
   • Bot Synonyms – Add and edit synonyms for the words in the bot or task names.
   • Entity Synonyms – Add and edit synonyms for any words in your dialog task Entity node names.
   • Field – Add and edit synonyms for any words in your task field names. Click each task to expand the word list for that task.

Adding Bot Synonyms

On the Synonyms page, on the Natural Language tab, you can view and add synonyms for words in bot task names. Synonyms defined on the Natural Language tab apply bot-wide to all task names, field names, and so forth.

Note that by default, synonyms are listed, this is limited to English, German, Spanish, and French languages alone.

To add a bot synonym

1. On the Synonyms page, on the Bot Synonyms tab, click Add Synonym. The New Keyword and Synonyms section is displayed.
2. In the Keyword field, enter the name of the word that you want to create synonyms for.
3. In the Synonym Words field, enter one or more synonyms for the Keyword. Press ENTER after entering each synonym.
4. Click Save to save and close the dialog.

Adding Entity Synonyms

On the Synonyms page, on the Entity Synonyms tab, each dialog task entity is listed with any defined synonyms for that entity.


To add synonyms for a dialog task entity

1. On the Synonyms page, click the Entity Synonyms tab.
2. Click the dialog task entity that you want to add synonyms for.
3. Enter one or more synonyms for the entity and then press ENTER after entering each synonym.
4. Click Save to save and close the dialog.

Adding Field Synonyms

On the Synonyms page, on the Field Synonyms tab, each task is listed along with the number of fields, or entities. Synonyms defined on the Field Synonyms tab apply only to the selected task, and the specified field name for that task.


To add synonyms for task fields

1. On the Synonyms page, click the Field tab.
2. Under a task name, click the name of the task field for which you want to enter the synonyms.
3. Enter one or more synonyms and then press ENTER after entering each field.
4. Click Save to save and close the dialog.

Field synonyms are only applicable to the associated task for the bot. This means that the same word, for example, Status in one task can have different synonyms for the same field called Status in a different task.

While adding synonyms for the words in a task or task field can help the NLP interpreter select the intended task or task field, what if the user input for the task or task field does not contain any of the words in the task name or task field name?

For example, for the Create Lead task for the Salesforce bot, if the user inputs New Account Option, which means the same as Create Lead, but does not match any of the words or synonyms for the task name. In this case, you can create patterns for groups of words that match the task name or task field.

Custom Concepts

Concepts are clusters of related and synonymous terms that you want to be considered as a group identified by a single term.

For example, you might want to use the term fruit to imply either apple or orange or banana.

Defining Custom Concepts

You can define a new concept the same way as you would define a Bot Synonym by starting the word with a ~. For more details on the naming convention refer here.

You can also use emojis to define custom concepts and the platform can process the information, for example use :wave: for hello or :thumbsdown as a no and :thumbsup as a confirmation.

Concepts can be used in defining patterns. For example, the pattern buy ~fruit can be used to capture buy orange or buy banana.

Managing Patterns

To optimize accuracy and recognition of the NLP interpreter, you can create patterns for names for tasks and task fields.

While using synonyms is great for words used in the name, sometimes, users may refer to a task using slang, metaphors, or other idiomatic expressions.

For example, a task name might be Get Current Weather, but the user inputs, what’s happening with today’s rain situation. In this case, none of the words used in the task name are used, yet the input has the same meaning as Get Current Weather.

In these cases, using patterns can help to improve NLP interpreter accuracy.

When the NLP interpreter matches a synonym to one task or field, and a pattern to a different task or field, the pattern match is prioritized and used for positive recognition over the synonym match.

To learn more about best practices and tips for optimizing NLP, see the Natural Language Processing Guide.

For a quick guide towards the usage of patterns, see How to use Patterns.

Navigating to the Patterns Tab

1. Open the bot for which you want to add patterns.
2. Hover over the side navigation panel and then click Natural Language > Training.
3. Click the Patterns tab.
4. On the Patterns page, click one of the following tabs:
   1. Intent Patterns – Add or edit patterns for dialog task User Intent nodes.
   2. Entity Patterns – Add or edit patterns for dialog task Entity nodes.
   3. Field Patterns – Add or edit task field patterns, for example, Date, Location, or Name as user input. This option is available only if the bot has one or more action, alert or information tasks.
   4. Task Patterns (this option has been merged with Intent Patterns post v6.40 release) – Add or edit patterns for any bot tasks, for example, Get Weather.

Adding Patterns for Intents

You can define a pattern as a sentence that represents the task but does not actually contain any of the words in the name of the task.

Also, you can define where words should be ignored by the NLP interpreter to improve accuracy for recognizing the correct task.

For example, you can build a pattern like find* testable* value to recognize in the user input a sequence of words in order where the word find is followed by any number of words, followed by the word testable, and again followed by any number of words, which are finally followed by the word value. For this pattern, the user input can be Find me any testable integers corresponding to the value, which would match the pattern.

You can also use concepts to build patterns. For example, the pattern can be defined as buy ~fruit and the user input that would match this pattern would be buy orange.

In the Intents Patterns section, you can view, add, and delete patterns for dialog tasks.

To add an intent pattern

1. On the Patterns page, click the Intent Patterns tab,
2. Click an intent name to enter the patterns for it.
3. In the Type and Enter to add patterns field, enter one or more sentence patterns, and press ENTER between patterns.
4. You can reorder, edit or delete patterns using the appropriate icons.

Adding Patterns for Entities

In the Entity Patterns section, you can view, add, and delete patterns for dialog task Entity node names.

To add an entity pattern

1. On the Patterns page, click the Entity Patterns tab,
2. Click an entity name to enter the patterns for it.
3. In the Type and Enter to add patterns field, enter one or more sentence patterns, and press ENTER between patterns.
4. You can reorder, edit or delete patterns using the appropriate icons.

Adding Patterns for Fields

You can view, add, and delete patterns for task fields in the Field Patterns tab.

Note: This option is available only if the bot has one or more action, alert or information tasks.

To add a field pattern

1. On the Patterns page, click the Field tab,
2. Click a field name to enter the patterns for it.
3. In the Type and Enter to add patterns field, enter one or more sentence patterns, and press ENTER between patterns.
4. You can reorder, edit or delete patterns using the appropriate icons.

Adding Rules

Note: This feature has been added in ver7.0

You can use traits or context tags to define intent identification rules for better intent detection. You can add Traits or Context Tags and conditional rules using AND, OR operators.

Refer here to know more about Traits and Trait Association Rules and Context Tags.

Negative Patterns

On the Kore.ai Bots Platform, natural language engine attempts to identify a user input and match that user input to a task. You can modify additional advanced settings to enable negative patterns for intent detection.

This will be useful in filtering matched-intents which are part of user utterance but should not trigger that intent.

Use Case: For example, a user says “I was trying to Book a Flight when I faced an issue“. Though the machine identifies the intent as “Book a Flight“, that is not what the user wants to do. In such a case, defining “was trying to *” as a negative pattern, would ensure that the matched intent is ignored.

Negative patterns can be used to eliminate intents detected by the Fundamental Meaning or Machine Learning models.

To enable Negative Patterns:

1. Go to Natural Language > Training
2. Select Advanced Settings
3. Enable Negative Patterns, by default this option is disabled.
4. You will see Negative Patterns section added, in this section:
   
   1. For each of the intent in your Bot you can define negative patterns similar to how you define intent patterns.
   2. Any intents identified by FM or ML engines, if identified as one of the configured negative patterns, will be ignored and not sent to Ranking & Resolver.
   3. Once added, these patterns can be reordered, edited, and deleted.

   

Threshold & Configurations

To train and improve the performance Threshold and Configurations can be specified for all three NLP engines – FM, KG and ML. You can access these settings under Natural Language > Training > Thresholds & Configurations.

The settings for FM engine is discussed in detail in the following sections.

NOTE: If your Bot is multilingual, you can set the Thresholds differently for different languages. If not set, the Default Settings will be used for all languages. This feature is available from ver 7.0 onwards.

Navigating to Thresholds & Configurations

1. Open the bot for which you want to configure thresholds.
2. Hover over the side navigation panel and then click Natural Language -> Training.
3. Click the Thresholds & Configurations tab.
4. Fundamental Meaning section allows you to set the threshold for FM engine:
   • Proximity of Probable Matches – this setting has been moved to under Ranking & Resolver, see here for details.
   • Intent Detection using Task Name Words (introduced in ver 7.1) can be disabled in case your use case requires it. This is particularly useful if you have too many tasks named – place order, cancel order, request order, duplicate order. All these tasks will be matched as intent if the user utterance has ‘order‘ in it. Instead of trying guess and train for all possible utterances, disabling this option would do the trick. Disabling it will not affect the intent detection using patterns, though. Also, an implicit pattern using the words in the Task Name will be automatically added. The implicit pattern will identify this intent only when the user input starts and ends with the exact task name. By default, it is enabled.
     

Suggested Reading
Know more about the guidelines in naming Intents, using Patterns and more, refer here.