Webchat -default text analyzer with pyNER text extraction model unable to detect indian names given in English as Person entity
I am using default pyNER text extraction model in Webchat default text analyzer for detecting name as Person.
By default Language setting,It doesn't detect Indian names given in English as Person entity.US names seems working fine.
1.Text analyzer -Language detection setting is Automatically detect language
examples-
a) Niranjan (Language -Croatian ,no entity detected)
b)Niranjan kumar (Language -English ,Person-kumar)
c)Rekha (Language-Swedish,no entity detected)
d)Rekha kumari (Language-Turkish,no entity detected)
e)Mike (Language-english ,Person-Mike ) -works fine
f)Mike Smith (Language-english ,Person-Mike Smith) -works fine
g)Mike smith (Language-english ,Person-smith)
h) James (Language-english ,Person-James) -works fine
i)mike (Language-english ,Person-mike) -works fine
2.Text analyzer -Language setting-Specify Language -English - It works expect for scenario b and g
Is it expected behavior as I don't want Webchat channel text analyzer to be constrained by Language setting.
There is setting "Language detected by publisher" that too works like default setting "Automatically detect language"
Updated: 22 Nov 2020 23:52 EST
Pegasystems Inc.
IN
Hi,
1. In multiple cases, the language detection is detecting other languages as there is not enough context around the names for determining the models. Even in the case when language is detected, the models are trained with additional context around the names. Please test with adding some more words and make a sentence so that model gets additional context around the entity. Typically 4-6 words should do. If still the name is not detected, you can give feedback to the pyNER model and update it.
2. "Language detected by publisher" option is not supported by the chat bot as of now.