Question
Pegasystems Inc.
US
Last activity: 23 Mar 2021 11:11 EDT
Negative/Neutral Response Configuration
I am looking for some guidance on the best way to configure negative responses for an outbound email adaptive model rule. I would like to modify the OOTB Email_Click_Through_Rate model that is associated with the Predict Outbound Email Propensity prediction, or just the prediction, if possible.
In our client's configuration, an external system sends the actions by email that are selected by CDH daily. This external system has its own rules that are similar to CDH channel limits, only they are more nuanced. The system identifies emails by category and has specific caps on the number of emails from each category that can sent each day.
Additionally, the system manages email campaigns for use cases that are not available in CDH. On any given day, a non-CDH campaign can trigger an email that results in a cap in the external system on the same category of email that CDH happens to select for a customer. This behavior is totally outside of our control. In these cases, a Pending interaction is logged in IH for that action, but CDH never receives a Send or Click response, because the email was suppressed by the external system's rules. This could also happen if the external system has an unsubscribe status that hasn't yet been ingested by CDH.
I am looking for some guidance on the best way to configure negative responses for an outbound email adaptive model rule. I would like to modify the OOTB Email_Click_Through_Rate model that is associated with the Predict Outbound Email Propensity prediction, or just the prediction, if possible.
In our client's configuration, an external system sends the actions by email that are selected by CDH daily. This external system has its own rules that are similar to CDH channel limits, only they are more nuanced. The system identifies emails by category and has specific caps on the number of emails from each category that can sent each day.
Additionally, the system manages email campaigns for use cases that are not available in CDH. On any given day, a non-CDH campaign can trigger an email that results in a cap in the external system on the same category of email that CDH happens to select for a customer. This behavior is totally outside of our control. In these cases, a Pending interaction is logged in IH for that action, but CDH never receives a Send or Click response, because the email was suppressed by the external system's rules. This could also happen if the external system has an unsubscribe status that hasn't yet been ingested by CDH.
In those cases, the OOTB adaptive models will log an automatic negative response after the learning window expires, which I think is inappropriate. The ideal behavior would be to set a learning window, capture any Send or Click responses during that time, and if neither response is captured, log NoResponse (or don't log any outcome at all).
Does this make sense? What is the best way to configure this in 8.5? I don't see any option to designate neutral responses.
A simple solution might be to set the response timeout on the prediction to Indefinitely and update the negative outcome on the adaptive model rule to Send. If I do this, should I be concerned about the volume of data being stored?
Accepted Solution
Updated: 23 Mar 2021 10:56 EDT
Pegasystems Inc.
NL
Setting a response timeout is generally recommended for the email use case as it defines the period in which a positive response can be received and automatically emits a negative when no response has been received. For reference: https://community.pega.com/knowledgebase/articles/decision-management/85/customizing-predictions
However, I understand that in your case, there’s an outside system that may block the sending. And if that happens when the make decision is already taken, then those will all end up to be negative (falsely, because for those there was no send).
As the Pega implementation doesn’t know which emails are sent, the only solution is to use the wait ‘Indefinitely’ option as you also suggest. There’s no real concern on the volume as the pxDecisionResults in Cassandra will expire on the specified TTL. However, you have to take care that the positive and negative responses to a campaign are sent to ADM in a mixed order. It shouldn't be all positives first (email clicks) and then all negatives or vice versa. A possible solution approach could be to store all pending responses and when a pos/neg is known for each (when the waiting period has expired), then feed the responses to ADM in the order of the interaction ID, which ensures they are mixed.
Regards,
Ivar
Pegasystems Inc.
US
@Ivar_Siccama Thank you for that feedback. Can you elaborate on the impact of not sending responses in a mixed order? We're currently experiencing an issue where positive responses aren't being attributed to our email models, and I'm wondering if this could be a contributing factor. (We opened a ticket INC-167730.)
Pegasystems Inc.
NL
@gaudc Not sending in a mixed order but e.g. all negatives first results in bad models - symptoms include seeing only 1 active predictor. Prior to the introduction of the time-out mechanism this was one of the most common problems when using adaptive in outbound.