Question
Future Proof AI
US
Last activity: 27 Aug 2022 19:12 EDT
Champion Challenger A/B testing
Has anyone implemented a champion challenger framework to do A-B testing in the NBA Strategy Framework?
At my current client there is a requirement to do A/B testing on different components of the NBA Strategy framework : eg Champion Challenger on Treatments or Actions or different engagement rules?
What are the best practices and where in the NBA Strategy framework is best to update this? At the moment we don't have a high level requirement from business however we want to implement and explore the capability of the champion challenger component.
In my experience with Pega and CDH so far I did not use this component rather than seeing it in Pega Academy course.
Any advice or suggestions will be greatly appreciated.
Regards,
Silviu
Pegasystems Inc.
US
@SilviuStefan , with the strategy framework in place, all the learning and experimentation is prebuilt for actions and treatments and there is little use for champion challenger usage. Since all the learning/experimentation is happening through the models the exception to the use of champion/challenger is likely to test different models, but even this is a pretty advanced use case, so unless you have very strong business requirements, I’d rely on the existing learning within the framework and not worry about champion/challenger.