In the case of multi-lingual bots, while much of the bot behavior does not change with the language, but the language semantics does impose some restrictions. In this document we will list out the different behaviors that would be language-dependent, so you as bot developer can cater for such scenarios.
Pattern Support
We have seen in this document, how Patterns can be used for Intent detection and Entity extraction. Those rules are for bots in the English language. Though the multilingual bots honor those rules, the language semantics impose some restrictions.
Also, patterns can also be used in Small Talk (while defining queries), Knowledge Graph (using alternate question field), and Trait identification but their behavior might differ slightly.
| Pattern Syntax | Intent detection and Entity extraction | Small Talk |
Traits | Knowledge Graph |
|
|---|---|---|---|---|---|
| Supported in Languages | Not Supported in Languages | ||||
| word1 word2 … wordn This mandates all the words defined to be available in the user utterance in the same consecutive order with additional words allowed in between, before and after |
All languages | – | Yes | Yes | Yes |
| word1_word2 Enforce phrase, no additional words allowed in between word1 and word2. |
All languages | – | Yes | Yes | Yes |
| word1 * word2 0 to infinite number of additional words between the specified words/phrases |
All languages | – | Yes | Yes | Yes |
| word1 *n word2 Exactly n number of additional words between the specified words/phrases |
All languages | – | Yes | Yes | Yes |
| word1 *0 word2 To disable wildcards between two tokens. |
All languages | – | Yes | Yes | Yes |
| word1 < word2 Indicates the match for word2 should start from the beginning of a sentence. |
All languages | – | Yes | Yes | Yes |
| word1 > word2 Indicates the end of the sentence and no words are allowed after it. |
All languages | – | Yes | Yes | Yes |
| !abc Indicates the word/concept “abc” should not exist anywhere in the user utterance after this token |
All languages | – | Yes | Yes | Yes |
| !!abc The very next word/concept should not be “abc” |
All languages | – | Yes | Yes | Yes |
| [ … ] Used to define a group of words/concepts and the match should be against exactly one of the group declared in [ ]. |
All languages | – | Yes | Yes | Yes |
| { … } Used to define an optional group or words/concepts and the match would be against zero or one of the words/patterns declared in { }. |
All languages | – | Yes | Yes | Yes |
| ( … ) contain a pattern i.e when a pattern or part of a pattern is enclosed in these parentheses, we treat it as a pattern unlike [ ] and { }. |
All languages | – | Yes | Yes | Yes |
| << … >> Used to find words in any order |
All languages | – | Yes | Yes | Yes |
| ‘word1 If you quote words or use words that are not in canonical form, the system will restrict itself to what you used in the pattern |
English, German, Spanish, French |
Portuguese(Brazilian), Italian, Chinese simplified, Chinese traditional, Indonesian, Korean, Dutch, Japanese, Arabic, Finnish, Russian, Polish, Ukrainian, Swedish, Kazakh, Norwegian, Hindi, Telugu, Tamil, Marathi, Catalan |
No | No | No |
| System Concepts Support for built-in concepts |
English, German, Spanish, French |
Portuguese(Brazilian), Italian, Chinese simplified, Chinese traditional, Indonesian, Korean, Dutch, Japanese, Arabic, Finnish, Russian, Polish, Ukrainian, Swedish, Kazakh, Norwegian, Hindi, Telugu, Tamil, Marathi, Catalan |
No | No | No |
| Developer concepts Support for developer-defined concepts |
English, German, Spanish, FrenchPortuguese(Brazilian)*, Italian*, Chinese simplified*, Chinese traditional*, Indonesian*, Korean*, Dutch*, Japanese*, Arabic*, Finnish*, Russian*, Polish*, Ukrainian*, Swedish*, Kazakh*, Norwegian*, Hindi*, Telugu*, Tamil*, Marathi*, Catalan* |
– | Yes | Yes | Yes* |
| Two-letter identifiers for Dates Like Mo for Monday, Tu for Tuesday, etc |
English, German, Spanish, Portuguese(Brazilian), Italian, Chinese simplified, Chinese traditional, Indonesian, Korean, Dutch, Japanese, Arabic, Finnish, Russian, Polish, Ukrainian, Swedish, Kazakh, Norwegian, Hindi |
French, Telugu, Tamil, Marathi, Catalan |
– | – | No |
* in these cases, developer concepts are not supported in entity extraction
NLP Support
Here we will list out the NLP features and their support.
| NLP Feature | Supported in Languages | Not Supported in Languages | Small Talk | Traits | Knowledge Graph |
|---|---|---|---|---|---|
| List of Values | |||||
| Full Match | All languages | – | – | – | No |
| Partial Match | All languages | – | – | – | No |
| Exact Match | All languages | – | – | – | No |
| Ambiguity behavior | All languages | – | – | – | No |
| Ambiguity with multi-item selection | English | Not supported in any language except English | – | – | No |
| Spell Correction | English, German, Spanish, French, Portuguese(Brazilian), Italian, Indonesian, Korean (from ver7.3), Dutch, Arabic, Finnish, Russian Polish, Ukrainian, Swedish, Kazakh, Norwegian |
Chinese simplified, Chinese traditional, Japanese, Hindi, Telugu, Tamil, Marathi |
– | – | Yes |
| Amend Entities | English | Not supported in any language except English | – | – | Yes |
| Sentiment Analysis | English, German, Spanish, French, Dutch, Korean (from ver7.3), Japanese (from ver8.0), Russian (from ver8.0), Kazakh (from ver8.0), Polish (from ver9.2), Norwegian |
Portuguese(Brazilian), Italian, Chinese simplified, Chinese traditional Indonesian, Arabic, Finnish, Ukrainian, Swedish, Hindi, Telugu, Tamil, Marathi, Catalan |
– | – | Yes |
| Bot Synonyms (default) | English, German, Spanish, French |
Portuguese(Brazilian), Italian, Chinese simplified, Chinese traditional Indonesian, Korean, Dutch, Japanese, Arabic, Finnish, Russian Polish, Ukrainian, Swedish, Kazakh |
– | – | Yes |
| Knowledge Graph | |||||
| Patterns | All Languages | – | – | – | Yes |
| Search in Answer | English, German (from ver7.3), Spanish (from ver7.3), French (from ver7.3), Portuguese(Brazilian) (from ver7.3), Italian (from ver7.3), Indonesian (from ver7.3), Dutch (from ver7.3), Finnish (from ver7.3), Polish (from ver7.3), Swedish (from ver7.3), |
Chinese simplified, Chinese traditional, Korean, Japanese, Arabic, Russian, Ukrainian, Kazakh |
– | – | Yes |
| Spell Correction | English, German, Spanish, French, Portuguese(Brazilian), Italian, Indonesian, Korean, Dutch, Arabic, Finnish, Polish, Swedish, Ukrainian Russian (from ver8.0), Kazakh (from ver8.0), Japanese (from ver8.0), |
Chinese simplified, Chinese traditional, Catalan |
– | – | Yes |
| Special Character support | All languages except English | English | – | – | Yes |
| NL Engines | |||||
| Dependency Parser Model | German (from ver7.3), French (from ver7.3), Italian (from ver8.0) |
English, Spanish, Portuguese(Brazilian), Chinese simplified, Chinese traditional Indonesian, Korean, Dutch, Japanese, Arabic, Finnish, Russian Polish, Ukrainian, Swedish, Kazakh, Norwegian, Hindi |
– | – | – |
Universal Bot Support
Here we will list out, language-wise, the trigger phrases used in training the Universal bot (see here) in the supported languages.
| Phrase | Language Support | ||||||
|---|---|---|---|---|---|---|---|
| English | German | Spanish | French | Portuguese (Brazilian) |
Italian | Chinese Simplified |
Chinese Traditional |
| Ask | Fragen | Pedir | Demander | perguntar | Chiedi | 问 | 問 |
| Tell | Sagen | Contar | Dire | Contar | Raccontare | 告诉 | 告訴 |
| Search | Suche | Buscar | Chercher | Procurar | Ricerca | 搜索 | 搜索 |
| Open | Öffnen | Abierto | Ouvert | Aberto | Aperto | 打开 | 打開 |
| Load | Belastung | Carga | Charge | Carga | Caricare | 加载 | 加載 |
| Begin | Start | Empezar | Commencer | Início | Inizio | 开始 | 開始 |
| Launch | Starten | Lanzamiento | lancement | Lançamento | Lanciare | 发射 | 發射 |
| Talk to | Reden mit | Hablar con | Parler à | Falar com | Parlare a | 交谈 | 交談 |
| Run | Lauf | correr | Courir | Corre | Correre | 跑 | 跑 |
| Exit | ausgang | salida | sortie | saída | uscita | 出口 | 出口 |
| Phrase | Language Support | ||||||
|---|---|---|---|---|---|---|---|
| English | Indonesian | Korean | Dutch | Japanese | Arabic | Finnish | Russian |
| Ask | Meminta | 물어보기 | Vragen | 頼みます | يطلب | Kysyä | Спросить |
| Tell | Menceritakan | 텔 | Vertellen | 伝えます | يخبار | Kertoa | сказать |
| Search | Cari | 검색 | Zoeken | 探す | بحث | Hae | Поиск |
| Open | Buka | 열다 | Open | 開いた | افتح | Avata | открыто |
| Load | Beban | 하중 | Laden | 負荷 | حمل | Ladata | нагрузка |
| Begin | Mulai | 시작 | Beginnen | ベギン | ابدأ | Alkaa | Начать |
| Launch | Meluncurkan | 시작하다 | Lancering | 打ち上げ | إطلاق | Tuoda markkinoille | запуск |
| Talk to | Berbicara dengan | 로 토크 | Praten met | に話す | تحدث الي | Puhua | Говорить с |
| Run | Lari | 운영 | Rennen | ラン | يركض | Juosta | Запустить |
| Exit | keluar | 출구 | uitgang | 出口 | خروج | poistuminen | выход |
| Phrase | Language Support | |||
|---|---|---|---|---|
| English | Polish | Ukrainian | Swedish | Kazakh |
| Ask | Zapytać | запитати | Fråga | сұрау |
| Tell | Powiedzieć | сказати | Säga | Tell |
| Search | Szukaj | Пошук | Sök | іздеу |
| Open | otwarty | відчинено | Öppna | ашық |
| Load | Załaduj | навантаження | Ladda | жүк |
| Begin | Zaczynać | почати | Börja | Баста |
| Launch | Uruchomić | запуск | Lansera | ұшыру |
| Talk to | Mówić do | Розмовляти з | Prata med | Сөйлесу |
| Run | Biegać | пробіг | Springa | жүгіру |
| Exit | wyjście | вихід | utgång | шығу |