Discussion
Pegasystems Inc.
US
Last activity: 2 Jun 2025 16:15 EDT
Ask the Expert - BIX Batch and Real-Time Data Extraction - Chaitra Kamaraju & Sahil Aggarwal
Welcome to our Ask the Expert session on BIX Batch and Real-Time Extraction) - Chaitra Kamaraju & Sahil Aggarwal
Make sure to Favorite and Follow for updates via the Notification bell and/or weekly digest!
Session dates: 26 May - 30 May
Session opens to questions on 26 May!
Meet Your Expert(s):
Chaitra Kamaraju and Sahil Aggarwal are Product Managers in Data & Integration area of the Pega Infinity. Chaitra has been with Pega for 14 years and in this journey, he played a crucial role in building product capabilities across various areas of the Platform. He along with Sahil are driving the evolution of Business Intelligence Exchange (BIX).
Sahil joined Pega in September 2025. He brings a total of 13 years’ experience with leading product companies delivering high-quality products.
Message from Your Expert(s): Chaitra Kamaraju and Sahil Aggarwal
We are excited to learn about how you leverage BIX today. These insights will guide us in enhancing the data extraction experience. Feel free to ask about what’s new in BIX and share how you are using BIX extract rules for your use-cases.
Ask the Expert Rules
- Follow the Pega Support Center's Rules of Engagement
- This is not a Live Chat - Chaitra & Sahil will reply to your questions over the course of the session
- Questions should be clearly and succinctly expressed
- Questions should be of interest to many others in the audience
- Have fun!
Areteans Technology Solutions
GB
@MarissaRogers I’ve been trying to configure real-time BIx extraction for my case type, but it doesn’t seem to be working as expected.
I’ve confirmed the following:
- The real-time extraction toggle is ON.
- The "RealtimeExtract" trigger is firing correctly when a case is created.
- The Queue Processor is running without issues.
- The Kafka connection is successfully established.
However, when I browse the Kafka dataset configured in the Extract rule, there are no records appearing.
I came across an article that suggests the first extraction might need to be a batch extraction. Could that be a prerequisite for real-time extraction to function properly? Or is there something else I might be missing in the configuration?
Any guidance would be appreciated.
Pegasystems Inc.
IN
@Arunkum@r Check if EnableDataExtraction release toggle is enabled.
You need to open DevStudio > Configure > System > Release > Toggles.
In the Search, provide EnableDataExtraction and do a Search.
Check if the Status is ON.
If the Status is not ON, perform an Edit using the pencil icon for that Toggle and select "Toggle for all"
Areteans Technology Solutions
GB
@ChaitraKVL As I mentioned in the previous message the toggle is ON.
Pegasystems Inc.
IN
@Arunkum@r Can you try to restart your system by clearing the in-memory cache.
We observed that in few scenarios when real-time extraction is configured, the DataSet configuration is not getting fetched in runtime.
Areteans Technology Solutions
GB
@ChaitraKVL Let me try.
Pegasystems Inc.
IN
@Arunkum@rI came across a post where you mentioned about this same issue.
What is the environment deployment type and the Kafka?
Have you customized the DataSet by including any Data Transform or Keys?
Have you tried to move the case further in processing, beyond the New stage?
Areteans Technology Solutions
GB
@ChaitraKVL Yes,
I am using Pega version 24.2.1, hosted in a Pega Labs instance. I am using pzInsKey as the key in the data set. Yes, I tried to process it end-to-end, but I still did not see any data in the data set.
Please find attached the screenshot of the data set and the extract rule.
Dataset
Extract
Pegasystems Inc.
IN
Can you please confirm if the BIX extract rule is configured with the data to be extracted.
Also, can you check execution runs for pzProcessBIXExtractEvents job (in Admin Studio) and any errors/exceptions reported in the logs.
Areteans Technology Solutions
GB
@SahilAggarwal @ChaitraKVL I found the root cause of my issue: my application was not added to the System Runtime context 😭 , which is required for the standard queue processor activity "pzProcessChangeDataCaptureCaseTypes" invoked by the real-time extract Declare Trigger rule. Thank you so much for the support.
Areteans Technology Solutions
GB
@SahilAggarwal @ChaitraKVL Sharing for others.
I tested Pega BIX real-time extraction by creating a cluster and topic in Confluent Cloud, and it worked very well. However, I noticed an issue: whenever I delete an existing real-time extract rule in Pega and create a new extract rule, the Declare Trigger rule is not created automatically. I have to create it manually and reference the trigger activity "pzQueueCaseForExtraction".
Maantic Inc
US
@MarissaRogers do we have any hotfix/patch to use this feature in pega 24.1.2?
Pegasystems Inc.
IN
BIX batch mode is available to use in Pega 24.1.2
BIX real time extraction is available from 24.2 onwards
Pegasystems Inc.
US
Thank you for checking out this Ask the Expert session!
Check out Ask the Expert: Authentication Best Practices with Sudhakar Tentu open now to questions!
Thank you to our experts, @ChaitraKVL & @SahilAggarwal