Question
Verizon
IN
Last activity: 14 Jun 2021 10:59 EDT
Evidence, Performance and propensity
What is the difference between Propensity, Evidence and Performance in Adaptive model.?
Model Evidence and Model Performance are the actual outputs of the adaptive model (set on the Output mappings tab), pyPropensity is the main model output. This is not clear in pega academy tutorial. Can I have one example?
Accepted Solution
Updated: 14 Jun 2021 10:59 EDT
Verizon
IN
@Otto_Perdeck These are general questions only. If everything works correctly, then recorded responses count in the model management page is same as the Model Evidence count of Adaptive model in a strategy. Thanks.
Pegasystems Inc.
NL
Propensity = likelihood of a positive response given the current set of predictor values. Value varies between 0.0 and 1.0.
Evidence = total number of positive and negative feedback the model has received thus far. Values start at 0 and can range into the millions or more.
Positives = number of positive feedback the model has received thus far. Values start at 0 and can range into the millions or more.
Performance = current model predictive performance expressed as AUC. Value varies between 50 (random) to 100 (perfect prediction - should not happen).
So, evidence, positives and performance are independent of the current set of predictor values, independent of the who or what you apply the model to. They say something about the state of the model.
Verizon
IN
@Otto_Perdeck Thank you. If I run the model after 10 responses do I get model evidence as 10 in the next strategy run? Can I consider Model Evidence = No of Responses received? Please clarify.
Pegasystems Inc.
NL
Yes. But. First of all the responses are only attributed to the model if everything is wired correctly - most importantly the labels in the response must match the outcomes defined on the model. Also, the response count may not be updated immediately, that depends on some settings in the model and service configuration.
Are these just clarifications or is there an issue behind these questions? If so better raise a support ticket with Pega or review the troubleshooting guides on community like this https://community.pega.com/knowledgebase/articles/decision-management/general-tips-troubleshooting-adaptive-models.
Thanks
Otto
Accepted Solution
Updated: 14 Jun 2021 10:59 EDT
Verizon
IN
@Otto_Perdeck These are general questions only. If everything works correctly, then recorded responses count in the model management page is same as the Model Evidence count of Adaptive model in a strategy. Thanks.
Pegasystems Inc.
NL
@JasmineM4474 Correct, thanks for the questions