Question
Areteans
AU
Last activity: 11 Nov 2018 6:51 EST
NLP - Stemmer and Lemmatiser
- Do the models built in Pega(like Max Entropy, Naïve Bayes, SVM) support Stemmer and Lemmatisation? If yes, is it inbuilt in Pega or do we have an option choose to have either Stemmer or Lemmatisation for the models.
- SVM and Max Entropy models support Part Of Speech tagging. Is this supported in pega?
- Pega 8 supports summary extraction feature. What is the logic that is used to extract the summary?
Pegasystems Inc.
US
Currently, only English language supports Lemming. Reference: https://community.pega.com/knowledgebase/articles/text-analytics-features-pega-7-platform
The ability to configure stemming and lemming is not available to the end business user and is inbuilt in the models we support.
POS tagging is a preliminary step and all models use POS feature sets as input. Again these are hidden from the end user.
Lastly, on text summarization, we have introduced this functionality with 8,1.
You can find all the details at
https://community.pega.com/knowledgebase/articles/extract-meaningful-summaries-text
Summary scores sentences based on the features and predictors. Higher weight is given to topics and facts detected in sentences. Top scored sentences are picked up as the summary.
regards,
Shantanu
Areteans
AU
Thanks for your reply Shantanu.
"POS tagging is a preliminary step and all models use POS feature sets as input. Again these are hidden from the end user. "
Can the model differentiate the meaning between "trust in a bank" and "you can trust in the bank". If yes, how is it reflected to the user? Is it the confidemce score?
In summary extraction, what does the topics relate to? Is it the topic determined by the model? In scenarios, where the text is completely irrelevant to any topics configured, how does the extraction functions?
Regards,
Veena