How to find the completion of Data Flow called from Activity
Hi,
We have a Dataflow called in an activity using DataFlow-Execute method, The activity calls the Dataflow in one step and before the Dataflow gets completed, the activity moves to the next step whereas the requirement is the activity should wait till the Dataflow completes the job and then activity should move to the next step.
How can we determine the dataflow completion status or is there any better way to deal this requirement?
***Updated by moderator: Lochan to add Categories***
Accepted Solution
Pegasystems Inc.
NL
There's currently no out of the box way of waiting for a batch data flow run completion, you would need to do that yourself. Usually batch data flow runs last for a very long time and because of that it is executed in a different thread so the current one is not blocked.
An alternative I can suggest is to use PostActivity processing. Once the data flow data processing is finished the configured activity will be triggered. Keep in mind that in this case the post activity will be executed in a different thread and the status of the run at that point will be "Running post activity".
If you need to wait for the data flow run to finish in the same thread you will need to implement the wait mechanism yourself. You can do that by invoking DataFlow-Execute GetProgress operation and verifying the value of pyStatus in the progress page. It should be either "Completed" or "Failed" once finished.
Hope that helps.
Henrique
Adqura
TR
Hi,
I've the same problem. I would like to wait for several seconds and check for the status of my DF. I do know that DF execution time is less than 10 sec.
In my activity Step-3 is where I call the DF. If I wait for ~20 seconds in step#4, How can I access the status of Step#3, where DF is called, when I am at Step #5 ?
Regards,
Pegasystems Inc.
NL
Hi cavlo,
As I suggested in a comment below, using Thread.sleep() is a risky workaround. What if you get to step #5 and the data flow hasn't finished yet? Consider using postActivity that queues another activity and continues processing. You can then load the progress of the data flow via DataFlow-Execute activity method and use "Get Progress" operation giving the run ID.
Adqura
TR
Then, please correct if I am wrong: Let following steps be my activity steps:
=================================================
Step #1 Log-Message (Activity starts)
Step #2 Call NewActivity (This activity calls my DF): and in this activity I'll use Operation "Start" and Run ID e.g. test-1
Step #3 DataFlow-Execute: This time Operation "Get Progress" and I can querry the status of test-1 by providing the Run ID
Step #4 Log-Message (Activity ends with success Or failure)
=================================================
Fuzzy points:
- In this design, we're sure that Step #3 will get executed when Step #2 is completed.
- Progress page stores the result of Test-1 run?
I just do not understand what makes sure that Step #3 will get executed after DF (post activity) completed.
Regards,
Oguz
Accepted Solution
Pegasystems Inc.
NL
There's currently no out of the box way of waiting for a batch data flow run completion, you would need to do that yourself. Usually batch data flow runs last for a very long time and because of that it is executed in a different thread so the current one is not blocked.
An alternative I can suggest is to use PostActivity processing. Once the data flow data processing is finished the configured activity will be triggered. Keep in mind that in this case the post activity will be executed in a different thread and the status of the run at that point will be "Running post activity".
If you need to wait for the data flow run to finish in the same thread you will need to implement the wait mechanism yourself. You can do that by invoking DataFlow-Execute GetProgress operation and verifying the value of pyStatus in the progress page. It should be either "Completed" or "Failed" once finished.
Hope that helps.
Henrique
Pegasystems Inc.
IN
Hi,
Glad to hear Siva has resolved the issue by himself. Further, He has shared the resolution in a private message. Here is the solution shared-
"Call the activity in pyPostActivity Property, The activity will be called only after Dataflow is executed."
Thanks!