Question
Rabobank
NL
Last activity: 17 Feb 2022 15:29 EST
Advantage of using function: sampleAdaptiveProportion(pos, neg)
In NBAStrategyFramework, the prioritization formula doesn't use propensity directly from adaptives - it's been driven by a function sampleAdaptiveProportion which takes 2 parameter, below complete formula.
@sampleAdaptiveProportion(1 + .pyPropensity * .ModelEvidence, 1 + (1 - .pyPropensity) * .ModelEvidence)
I tried running this expression multiple times with a propensity value and it was giving a random value b/w 0-1 each time.
Can someone please explain me in simpler terms what the above formula does? And what is the advantage of using it over smooth propensity which we used earlier? This will help me to explain it our business team.
P.S: I have no experience in data science, so a layman explanation would really help. Otherwise if there a good source to learn and understand then please refer so.
Accepted Solution
Updated: 25 Jan 2022 8:55 EST
Pegasystems Inc.
US
The function that you mention - sampleAdaptiveProportion - is used to apply Thompson sampling to the propensity, as it generates better results that propensity smoothing when applied to immature models.
Here are some articles that describe how Thompson sampling is used.
https://docs-previous.pega.com/pega-customer-decision-hub-user-guide/85/model-learning-new-actions
https://academy.pega.com/topic/configuring-adaptive-model/v1
Rabobank
NL
@hellc Thanks for responding!
Model maturity is only applicable for outbound channels and after reading documents you shared, I understand Thomson sampling should only be applied to outbound channel? if yes, then something doesn't add up because the formula where it has been implemented will also be applied for Inbound channels.
Next to that, is it really necessary to keep a strict value of 200 positive response for declaring a model as mature? I am asking this because there are few offers which are only applicable for let's say 5k customers or even less, so in those scenario 200 will be a big number. Shouldn't we have a flexibility to define a number at Action level? or maybe an option to skip model maturity if needed?
Pegasystems Inc.
US
@Gaurav_Kumar Thompson sampling and Model Maturity are two separate processes; Thompson sampling is applied to both inbound and outbound interactions since we need to adjust the propensity of all actions when the model evidence is low.
The value of 200 for Model Maturity may be overridden by overtaking the strategy and adjusting the value. If you want to define the value at action level you could add a custom property to the action form for that purpose, or use an existing 'unused' property.
Model Maturity can be disabled from the Next Best Action Designer settings (gear icon towards top right of screen).
Updated: 12 Jan 2022 15:30 EST
Rabobank
NL
@hellcThanks for clarification, since model evidence is not passed in the function (sampleAdaptiveProportion), can I assume it is internally being referenced? I am asking this question just to know how this function will get to know the evidence count.
Pegasystems Inc.
US
@Gaurav_Kumar Here's the call to the function showing that model evidence is part of the input:
@sampleAdaptiveProportion(1 + .pyPropensity * .ModelEvidence, 1 + (1 - .pyPropensity) * .ModelEvidence)
Rabobank
NL
@hellcyeah, I saw that, but those are calculations which will pass two 'double' values to the function, not the 'ModelEvidence' as property itself.
Anyway, I am going to add FinalPropensity (result of the function) to IH-Fact and later on analyse when (approx. model evidence count) it is actually getting converged to pypropensity.
Thanks for your help in explanation!
Pegasystems Inc.
US
@Gaurav_Kumar FinalPropensity should already be present in IH Fact
Accenture
US
@hellc What model evidence is considered as low? as I don't see any specific setting for that. Also, will thomson sampling eventually converge with the actual model propensity once it receives enough evidences?
Pegasystems Inc.
US
@ChandraKanumuru There's no strict definition of what constitutes low evidence. As in the original smoothed propensity formula, as the model evidence increases the function result will approach the actual propensity.
Accenture
US
@hellc Thanks Chris, for the clarification. Also, the model evidence is nothing but the number of responses captured by the model so far (positve+negative) ,correct? Is there a latency in updating the model evidence value , because currently our model evidence doesn't match with the responses captured/updated in the model neither with total model responses nor with the positive responses. I see difference in 1000's between the two values.
Pegasystems Inc.
US
@ChandraKanumuru I'd suggest creating a different post with the new topic. I also would have thought they should match up.