Question
NASCO
US
Last activity: 13 Sep 2017 4:45 EDT
BIX Extracting Duplicate records into DB table in single extract run
Hi
We are running BIX rule to load a DB table. We have a filter condition for extracting work object and do not check use last update time to run the extract.
Before running BIX everyday we truncate data from this table and reload using BIX.
We are seeing duplicate records in the table for the same pzInsKey with in the same run. We are thinking there may be updates happening with in the app for those kind of duplicate records
As per my understanding we need to get only 1 record per pzinskey that is most updated at that time.
Is this normal behavior of BIX or is there any HFix available to resolve this issue.
Pegasystems Inc.
IN
HI Aravind,
If the same work object is modified yesterday and today, it will be extracted twice (though you are using the "Use last updated time as start" option ).
Please refer to BIX design models page https://docs-previous.pega.com/business-intelligence-exchange-bix/bix-design-models . If you go down to the section "Designs for different table models", you will notice the delete query to use for "transaction table". This could be one way to handle this.
In future, we may handle this in the BIX product but currently there is no OOTB functionality to do this.
Hope this helps. Please let me know if you need any more details.
Thanks & regards,
Venkat
NASCO
US
Hi venkat,
Thanks for replying, our scenario here is not extracting same record on 2 different days but we are seeing same record twice on the same day in the same extract run. I understand pega only inserts data and if the record is already existing in the table previous day and next day it will insert 2nd record with same pzInskey.
We are not using "Use last updated time as start" option but we have our own query as a filter.
Regards
Aravind
Pegasystems Inc.
IN
Hi Aravind,
Please set logging level of class "internal.access.ExtractImpl" to debug and check bix.log.
It should show all the insert queries and from there you will vbe able to identify if duplicate inserts are happening at all.
You can also configure to run bix with output as CSV and verify if you get duplicate results there as well.
Thanks,
Ujjwal