Question
De Volksbank NV
NL
Last activity: 4 Dec 2023 8:52 EST
Data Set Performance Issue
We have noticed very inefficient processing from Data-Set-Execute in a specific scenario. The development server has three Util nodes. A Job Schedule running an activity on a single Util node runs a Data Page to get a set of records for deletion.
The records are from a Data- derived class. The class is mapped to a single database table.
An activity iterates over the data page record list and uses a Data Set to delete one record per iteration using the DataSet-Execute method and uses .pzInsKey as the key. (Activity fragment at bottom of page).
This process seems to generate a very heavy CPU loading on all three Util nodes:
Has anyone seen similar behaviour?
I have two theories:
We have noticed very inefficient processing from Data-Set-Execute in a specific scenario. The development server has three Util nodes. A Job Schedule running an activity on a single Util node runs a Data Page to get a set of records for deletion.
The records are from a Data- derived class. The class is mapped to a single database table.
An activity iterates over the data page record list and uses a Data Set to delete one record per iteration using the DataSet-Execute method and uses .pzInsKey as the key. (Activity fragment at bottom of page).
This process seems to generate a very heavy CPU loading on all three Util nodes:
Has anyone seen similar behaviour?
I have two theories:
- Either the set-up overhead of submitting to the Queue Processor is very heavy per invocation of DataSet-Execute
- OR... The single row deletes are being submitted to the database in large numbers in parallel - causing conflict on the DB as each needs a row-level lock. It could be escalating to a table lock and blocking other threads?
Activity fragment:
Pegasystems Inc.
GB
@RICHARDA2870 if you still need help with this, please log a support incident via the MSP.