Question
Coforge
US
Last activity: 28 Apr 2021 11:59 EDT
Dataflow with specific dataflow case id fails always
We are making use of DataFlow-Execute method with run options configured to create a Batch processing dataflow with a specific work object id.
So during our runs, one time there was an infrastructure issue with the dataflow nodes and one particular run through DataFlow-Execute failed it happened some time ago.
Till today the Run events are being generated with a stop, so now when our execution starts we get an event of stop and the dataflow fails to initialize. And this is causing issues for us for that specific ID. We have a workaround by making use of a different id for the failed run. But the ideal solution for us would be to find a way to clear out all the old events which are propagating message to stop so that new run doesn't fail.
Has anyone come across the same scenario? Any inputs would be highly appreciated. Thanks!
We are making use of DataFlow-Execute method with run options configured to create a Batch processing dataflow with a specific work object id.
So during our runs, one time there was an infrastructure issue with the dataflow nodes and one particular run through DataFlow-Execute failed it happened some time ago.
Till today the Run events are being generated with a stop, so now when our execution starts we get an event of stop and the dataflow fails to initialize. And this is causing issues for us for that specific ID. We have a workaround by making use of a different id for the failed run. But the ideal solution for us would be to find a way to clear out all the old events which are propagating message to stop so that new run doesn't fail.
Has anyone come across the same scenario? Any inputs would be highly appreciated. Thanks!
Accepted Solution
Coforge
US
Solution given by Pega:
To clear the DF run event messages, you may execute any of the following activities pyDeleteAllLogEvents, pyDeleteEventsOlderThanNDays and pyDeleteOldEvents.
These are the activities which can be used to delete events in Log-Decision-DataFlow-Event
In order to clear the DF run object and its related run event details; you may run the below query against the DB
- delete from pegadata.pc_work_dsm_batch where pyid = 'DF-1234';
- delete from pegadata.pr_data_decision_ddf_run_opts where pyworkobjectid = 'DF-1234';
- delete from pegadata.pr_data_decision_df_part where pxworkid = 'DF-1234';
- delete from pegadata.pr_data_decision_ddf_error where pxworkid = 'DF-1234';
- delete from pegadata.pr_data_decision_df_met where pxworkid = 'DF-1234';
- delete from pegadata.pr_log_dataflow_events where pyrunid = 'DF-1234';
- delete from pegadata.pr_data_decision_ddf_runtime where pyrunid = 'DF-1234';
Kindly use the relevant DF run ID in this query which you want to clear. Some the query on the above list may vary depending on the specific Pega 8.x version.
Updated: 24 Mar 2021 8:12 EDT
Coforge
US
So the event which is received by the dataflow which makes it stop as soon as it starts is as below:
| Status changed | dataflow-i-tesdfdfddd | dataflow-i-tesdfdteeeee | 3/24/21 12:06 AM | {"type":".RunStatusTransitionMessage","senderNodeId":"dataflow-i-ts343433333333","timestamp":1616562378933,"runId":"DF-<CaseID>","previousStatus":"COMPLETED","newStatus":"INITIALIZING","originator":"ClearState","reason":"Clear state for new run"} | 1.61656E+12 |
I am starting to believe since the dataflow starts the stop messages from the previous day's run are captured by this dataflow case as we are reusing the same caseid (DF-CaseID) that fails the dataflow sometimes during initialization.
Does anyone know how to clear event flow queue of dataflow for a specific runId?
So the event which is received by the dataflow which makes it stop as soon as it starts is as below:
| Status changed | dataflow-i-tesdfdfddd | dataflow-i-tesdfdteeeee | 3/24/21 12:06 AM | {"type":".RunStatusTransitionMessage","senderNodeId":"dataflow-i-ts343433333333","timestamp":1616562378933,"runId":"DF-<CaseID>","previousStatus":"COMPLETED","newStatus":"INITIALIZING","originator":"ClearState","reason":"Clear state for new run"} | 1.61656E+12 |
I am starting to believe since the dataflow starts the stop messages from the previous day's run are captured by this dataflow case as we are reusing the same caseid (DF-CaseID) that fails the dataflow sometimes during initialization.
Does anyone know how to clear event flow queue of dataflow for a specific runId?
Accepted Solution
Coforge
US
Solution given by Pega:
To clear the DF run event messages, you may execute any of the following activities pyDeleteAllLogEvents, pyDeleteEventsOlderThanNDays and pyDeleteOldEvents.
These are the activities which can be used to delete events in Log-Decision-DataFlow-Event
In order to clear the DF run object and its related run event details; you may run the below query against the DB
- delete from pegadata.pc_work_dsm_batch where pyid = 'DF-1234';
- delete from pegadata.pr_data_decision_ddf_run_opts where pyworkobjectid = 'DF-1234';
- delete from pegadata.pr_data_decision_df_part where pxworkid = 'DF-1234';
- delete from pegadata.pr_data_decision_ddf_error where pxworkid = 'DF-1234';
- delete from pegadata.pr_data_decision_df_met where pxworkid = 'DF-1234';
- delete from pegadata.pr_log_dataflow_events where pyrunid = 'DF-1234';
- delete from pegadata.pr_data_decision_ddf_runtime where pyrunid = 'DF-1234';
Kindly use the relevant DF run ID in this query which you want to clear. Some the query on the above list may vary depending on the specific Pega 8.x version.
Pegasystems Inc.
US
Thanks for sharing the resolution @imranullah!
Did you open a Support Case? If so, could I have the ID please for tracking?
Thank you!
Coforge
US
@MarissaRogers INC-168439