Question
JP Morgan Chase
US
Last activity: 4 Jun 2021 6:33 EDT
Adaptive Model AUC and Predictor AUC
Hi,
Can anyone explain me how the Model AUC is calculated? And How Predictor AUC is calculated.?
Regards
Indira
Pegasystems Inc.
NL
AUC is a standard metric in computer science and is calculated from the ROC curve. See e.g. https://en.wikipedia.org/wiki/Receiver_operating_characteristic.
Most ROC/AUC algorithms that you see use N datapoints with a propensity and a label. Since we don't have the N datapoints but only the aggregated bins for both the classifier and the predictors we calculate the AUC from these bins. Each bin represents #pos true positives with a propensity of #pos/(#pos + #neg) and #neg true negatives for which it gives that same propensity.
Hope that helps.
Swedbank As
EE
@Otto_Perdeck , I have the same question, but unfortunately the response above is not as detailed as I expected. Is there a concrete formula how predictor AUC are combined to get model AUC?
Example: overall model AUC is 86.27, active predictor’s (alltogether 59 predictors) scores vary from 52.31 to 94.499. Is there a formula how the model AUC weighted calculation is done?
Pegasystems Inc.
NL
Thanks for your question. Let me correct one thing first: model AUC is not calculated from the predictor AUC's.
The predictor AUC's indicate the univariate performance of the predictor (for that particular model). You can easily calculate that AUC from the binning of a predictor. Predictor AUC is an indication of the predictive power of that predictor but nothing more than that.
The model AUC is calculated from the binning that you see in the Score distribution tab of the model. The way the AUC is calculated from these bins is the same of course. The input to the score distribution is the cumulative log odds of the active predictors.
Hope that helps otherwise see if your Pega consultants on the team can do a deeper dive into Adaptive analytics.
Otto