parallel processing
Hi All,
we have below requirement which needs parallel processing or multi threading.
We have exposed a service rest to validate user account information in different Tables. like CreditCard, PersonalLoan and HomeLoan tables to send number of time user account holder failed to Pay bills/EMis ontime , so the Sample request that I would be receiving is
{
"AccNo":"123456789",
"DOB":"01-01-1988",
"CustID":"ABC123"
"FName":"ABCDEF",
"LastName":"XYZ",
"ValidationTypeList": ["CC","PL","HL"]
}
When I receive the above request i need to hit 3 different tables and 3 different validation logic for each validation type and all are independent so We are trying to implement parallel processing based on validation typeList instead checking one after the other, like one thread can go and check CreditCard Info Table, one another thread can check Personal Loan info and other can check HomeLoan in parallel and at the end we have to bundle all the 3 responses into one and send back the response to the requestor . Can you please let me know best approach . FYI : this service we provided as synchronus service
Thanks
Pegasystems Inc.
AU
Hi @NARENDRAD
Great question.
The approach I like to guide people towards for this is to use Asynchronous Data Pages.
Build each independent check into its own Data Page: one Data Page for credit card; one Data Page for home loan; one Data Page for personal loan.
Then your service activity can then use the Load-DataPage method to load each Data Page asynchronously. Put a When precondition on each Load-DataPage step to check your ValidationTypeList for which Load-DataPage steps should be performed / skipped.
Use the same PoolID for each Load-DataPage step to group together the logical set of Data Pages to load at the same time. Ideally this would be a value unique to the current service call (perhaps the current Requestor ID?), otherwise you risk multiple service calls asking for asynchronous loading bound to the same batch.
Use a Connect-Wait with the same PoolID at the later point in your activity immediately before you need results from one of the data pages. The activity blocks on this step until all the data pages from that pool have finished loading.
Asynchronous data page loading happens in different (child) Requestor to the activity that needs them. Ensure that all the inputs needed to load each Data Page are passed as parameters, for as they load in a different Requestor they won't have access to any other Clipboard state from your service activity whilst loading.
Hi @NARENDRAD
Great question.
The approach I like to guide people towards for this is to use Asynchronous Data Pages.
Build each independent check into its own Data Page: one Data Page for credit card; one Data Page for home loan; one Data Page for personal loan.
Then your service activity can then use the Load-DataPage method to load each Data Page asynchronously. Put a When precondition on each Load-DataPage step to check your ValidationTypeList for which Load-DataPage steps should be performed / skipped.
Use the same PoolID for each Load-DataPage step to group together the logical set of Data Pages to load at the same time. Ideally this would be a value unique to the current service call (perhaps the current Requestor ID?), otherwise you risk multiple service calls asking for asynchronous loading bound to the same batch.
Use a Connect-Wait with the same PoolID at the later point in your activity immediately before you need results from one of the data pages. The activity blocks on this step until all the data pages from that pool have finished loading.
Asynchronous data page loading happens in different (child) Requestor to the activity that needs them. Ensure that all the inputs needed to load each Data Page are passed as parameters, for as they load in a different Requestor they won't have access to any other Clipboard state from your service activity whilst loading.
Once the Connect-Wait is complete, the loaded data pages are made available to the current Requestor and referencing them just like you would reference any other data page will get you access to the asynchronously loaded result set. Don't forget to check your Data Pages for errors!
To debug, turn on the "ADP Load" event in the tracer (I think this is there in 7.3.1). This allows you to drill into each of the child Requestors used when the data pages are asynchronously loaded and see their tracer output. Really neat!
Carelon
IN
Hi BraamCLSA,
Thanks for the reply ,
I have tried the above approach and facing below issues
created 3 datapages but while mapping response from each datapage into may MyServicepage.Response , again datapage getting reloading and it is errored out as the request page is not available while reloading.
Can you please help me how to fix this issue ?
As you can see in attached activity steps , 6 and 7 steps are loading the data page which is working fine.
Step 8 Connect wait with same poolID
Step 9 & 10 setting response from Datapage to Myservicepage.response ( here datapage trying to load again which should not)
Can you please let me know solution for this issue ?
Thanks
Naren
Pegasystems Inc.
AU
Hi @NARENDRAD
Any time a Data Page is used as a Step Page to an Activity method, Pega is going to check whether that Data Page exists on the clipboard and (synchronously) load it, ensuring it has the context to perform the method you've configured for that step. This obviously isn't what you want in this case.
In steps 6 and 7, try moving the Data Page name from the Step Page into the parameter of the Load-DataPage method (adjacent to PoolID), and leaving the Step Page blank.
Expressing this in pseudo-code, you currently have:
D_RiderValidation.loadDataPage("", "MyPoolId")
... whereas instead, you want ...
Activity.loadDataPage("D_RiderValidation", "MyPoolId")
That is, consider "Load-DataPage" is a static call whose context is irrelevant. It does what it needs to do based on the parameters it is passed, and doesn't care about the Step Page.