Control group at proposition level with validity period
Hi, We are having a similar requirement of creating control groups at proposition level. The business directive behind this is to measure the effectiveness of the proposition launched via Pega ( inbound/outbound channels) against any other channels. Here are the high level requirements: Control group would be based on a target group which would be randomly selected and designated as the nth customer. For e.g. if there are 200 customers in the target group then the business would configure every 20th customer to be in control group ( like 20th, 40th, 60th etc till 200th). Further to this there would be a so-called blackout ( validity) period of 100 or 150 days. This means if a customer is in a specific control group based on the interaction across inbound and outbound channels, then he/she would continue to remain in that group for that period. So it means that the same 20th customer can become the 10th customer viewing / reviewing the proposition in his next interaction, he would still belong to the control group as long as the second interaction is happening within 100 or 150 days. The reverse is also true for a customer who is not in a control group would not be in a control group for that entire period of 100 or 150 days. We are kind of stuck in arriving at a potential solution for this, mainly because this is not in line with what Pega uses control groups for ( using segments and outbound marketing).
Hi, We are having a similar requirement of creating control groups at proposition level. The business directive behind this is to measure the effectiveness of the proposition launched via Pega ( inbound/outbound channels) against any other channels. Here are the high level requirements: Control group would be based on a target group which would be randomly selected and designated as the nth customer. For e.g. if there are 200 customers in the target group then the business would configure every 20th customer to be in control group ( like 20th, 40th, 60th etc till 200th). Further to this there would be a so-called blackout ( validity) period of 100 or 150 days. This means if a customer is in a specific control group based on the interaction across inbound and outbound channels, then he/she would continue to remain in that group for that period. So it means that the same 20th customer can become the 10th customer viewing / reviewing the proposition in his next interaction, he would still belong to the control group as long as the second interaction is happening within 100 or 150 days. The reverse is also true for a customer who is not in a control group would not be in a control group for that entire period of 100 or 150 days. We are kind of stuck in arriving at a potential solution for this, mainly because this is not in line with what Pega uses control groups for ( using segments and outbound marketing). There is also a challenge to apply the logic of the validity period on the proposition level. From a performance perspective ( esp in inbound channels), scanning through the entire IH_FACT to pick up if a customer has belonged to the control group in the past when we go live with this solution. Any ideas / thoughts / directions would be very helpful.
Pegasystems Inc.
NL
Hi,
Measuring effectiveness via control groups is always a good idea. I'm not intimately familiar with how Pega Marketing uses control groups but just reasoning from platform perspective: scanning IH_FACT is certainly not a good idea but is it possible for you to do control group assignment based on a hash code derived from the two elements customer ID and time?
From interaction time derive a number representing units of 150 days. Combine this with the customer ID then do a hash on both; 5% of these are the ones currently in the control group. This would guarantee consistent control group assignments for customers during periods of 150 days.
Cheers,
-Otto
Cognizant Technology Solutions
IN
Thanks Otto for your reply.
But could you explain this statement a bit more - "From interaction time derive a number representing units of 150 days". ?
Also when we do a hash on customer ID and Interaction Time it is certain that same customer will be repeated. So will 5% give me the unique set of customers ?
Thanks in advance.
Pegasystems Inc.
NL
Hi
Well maybe I'm putting you on a wrong path but this seems to (roughly) satisfy the requirements:
From the interaction time you get a number that is the same for 150 consecutive days. You would take the # of days since a certain epoch and integer-divide that by 150.
Concatenate this number to your customer ID. Then create a hashcode from this combined string (a simple call out to a function could do this - I can even look up some code to do that). If the returned range is 0..1000 (for example), then whenever that hash value is < 50 you would mark this customer at this time as being in the 5% control group.
Would that work for you?
Otto
Cognizant Technology Solutions
IN
Please find below my understanding of the solution:
Lets say there is a ValidityStartDate(epoch) as today. So tomorrow, the number to be concatenated to CustomerID comes as 1/150 = 0.0067
customerID = C1
Concatenated customerID=C10067
Hash Code = 45
If 45<50, then customer in control group.
Is my understanding correct?
Also, do we actually need the Interaction Time here?
Thanks,
Aniket
Pegasystems Inc.
NL
Just dont forget to do integer division (or truncate the resulting number). So 1/150 is truncated to 0, as well as 13/150. 152/150 --> 1 etc. Only that gives you consistent hashing for the same customer during a period of 150 days.
Pegasystems Inc.
NL
Hi
Please allow me to address the original statement "...this is not in line with what Pega uses control groups for ( using segments and outbound marketing)".
Control Groups in Pega Marketing is meant to be used across inbound and outbound channels. When you create a control group, you not only decide how customers are selected (absolute count or percentage), you can also define its validity period (exactly something you are looking for).
Information about the customers identified in the control group is written into Interaction History precisely because you can use it in the strategy to make decisions irrespective of the channel of interaction.
You do not have to worry about the performance of querying Interaction History from the strategy. It is always optimized and something handled within the product.
Please go through the Pega Academy learning module Measure Marketing Effectiveness Using Control Groups for more details on how it works.
I strongly recommend to use the out-of-the-box functionality as much as possible.
Regards
Guru
Pegasystems Inc.
NL
@Guru - will the validity period option also allow one to do the "rolling assigments", as this is what I read from the requirement? I.e. any user is part of the control group only during a number of days, then will automatically be swapped out and replaced by someone else who then enters the control group for that number of days?
Pegasystems Inc.
NL
Hi Otto
Yes. You can set-up the Segment to run at a particular frequency (once every 3 months for example). Assuming in the Segment options 'Random Sample' and 'Control Group' are set, every time the Segment runs, new set of customers will be part of the control group. Existing members will be marked as 'non-members' in the IH.
One caveat is that, due to randomness, same customers may once again be part of the control group. But they can be filtered out in the strategy as all past information about the control group membership is available in the IH. The Segment only adds new information to IH, never deletes it.
Regards
Guru