Importance of ADM predictors
Hi All,
I would like to understand how the importance of active predictors is calculated in the ADM model performance.
Ex: I have Predictor 1(Active, Latest Performance: .7), I have Predictor 2(Active, Latest Performance: .65), I have Predictor 3(Active, Latest Performance: .6)
So that means all these 3 predictors have equal importance in defining the ADM model output?
This is for the Pega ADM model based NB classifier not the new GB algorithm in 8.7 CDH.
Pegasystems Inc.
NL
Hi @Nizam
The performance of the predictors is the univariate performance: it is an indication of the correlation of this single predictor with the outcome. The univariate performance is used to deselect predictors that have a value below a threshold (.52). Other than that it is not used and only serves reporting purposes.
The performance of the model is not directly related. The active predictors are combined into the naive bayes model. The average log odds of all the predictor values is basically the index into the score-to-propensity mapping which you see in the Classifier of the model. The performance of the model is calculated from the classifier mapping.
Perhaps this helps: https://pegasystems.github.io/cdh-datascientist-tools/articles/adm-explained.html.
Regards
Otto
Tech Mahindra
IN
@Otto_Perdeck Thanks for the details.
If the performance of the model is not directly related to active predictors, then how can we improve the overall performance of ADM model.
Pegasystems Inc.
NL
Hi @Nizam
I'm not suggesting that the model performance is not related to the active predictors, of course it is, what I tried to convey is that the predictor AUCs are not directly used in the model.
The question is how to improve model performance is very generic. If there are specific questions behind this q I'd recommend working with your CSM for this particular situation. The generic answer is that there are two aspects to this
1) The input to the models. Make sure the (candidate) predictors are from a varied set of sources and follow the general best practices. See Academy and for example https://docs-previous.pega.com/decision-management/87/best-practices-adaptive-and-predictive-model-predictors.
2) The target of the models - what they're to predict. How predictable is this outcome and how is this measured?
Hi @Nizam
I'm not suggesting that the model performance is not related to the active predictors, of course it is, what I tried to convey is that the predictor AUCs are not directly used in the model.
The question is how to improve model performance is very generic. If there are specific questions behind this q I'd recommend working with your CSM for this particular situation. The generic answer is that there are two aspects to this
1) The input to the models. Make sure the (candidate) predictors are from a varied set of sources and follow the general best practices. See Academy and for example https://docs-previous.pega.com/decision-management/87/best-practices-adaptive-and-predictive-model-predictors.
2) The target of the models - what they're to predict. How predictable is this outcome and how is this measured?
The following 2 event series may be good to be aware of: https://collaborate.pega.com/discussion/cdh-community-events https://collaborate.pega.com/discussion/cdh-enablement-series In the first series (Community Events) there was a very good session in November 2021 (scroll down to past events) that described the implementation of adaptive at a client, including all the learnings. Also, the december session was about data science.
For troubleshooting I recommend taking a look at https://docs-previous.pega.com/decision-management/87/troubleshooting-adaptive-decision-manager-service. Hope this helps, -Otto
Updated: 29 Nov 2022 7:26 EST
Cap Gemini
GB
@Otto_Perdecksite not found