Closed
Machine learning text categorization model topic confidence value
In Machine learning text categorization model, when model picks the topic, it picks more than 1 topic with different confidence score, what threshold value is good enough for confidence score to make some decision based on that ?
***Edited by Moderator Marissa to update General to Product***
Posted: 4 years ago
PEGA
Pegasystems Inc.
IN
It's hard to say what threshold value is ideal because it depends on individual use cases. Mostly it depends on what is the tolerance that business expects. If confidence threshold is set at -
- 80% - Manual triaging will have to be high initially because initially models will have ~70 confidence.
- 50% - Very aggressive, business wants to automate as many emails as possible and is okay with few False Positives (Emails categorized wrongly)
- 90% - Not recommended initially. As more feedback comes in from real world usecases, model will become more and more confident and threshold can be slowly increased.
In nutshell - The lesser the confidence threshold, higher will be the automation rate but higher will be False Positives