Question
US
Last activity: 16 Jan 2026 8:45 EST
Is Asynch processing using Load-DataPage approach scalable?
I have implemented a Service REST API which executes business rules paralelly using Load-DataPage, Connect-Wait approach.
Request to this API will have a lot of volume where multiple requests may hit the Pega API endpoint every second. So wanted to understand
1. What are the configurations that need to be checked in Pega related to this implementation? E.g. Is there any threshold on the number of parallel requestors that can be spawned to handle the requests? Any configurations to verify in Infra?
2. Is this implementation approach scalable to handle a high volume of requests?
The application is hosted on Pega Cloud and uses Pega Infinity '24.2.3.
Pegasystems Inc.
CA
Though Load-Datapage call async but in conjuction with Connect-Wait, it might impact the user expeerience because you're waiting for couple of seconds at least until the response is received.
Can you think of Queue Processor in your case? This will be indepdent and minimize the user impact. This will help you to monitor the load and increase the number of threads based on your use cases.
Thanks,
Ramesh
US
@RameshSangili Thanks for sharing your thoughts.
QP is not a possible option here as the application(s) invoking the Pega API need a synchronous response i.e. will wait for the response in the same invocation.
biG
IN
Option#1: Better to use Kafka topic as messaging means push the request in kafka queue and read it by dataset , process with Data flow.
It would be processed million of records records with very less time because it will use many partitions to process the request.
option#2 :Try to execute asynchronously (queue for execution) approach in service Rest and increase the no. of nodes.