Question
CNO
US
Last activity: 12 Nov 2019 11:55 EST
Design for dealing with the large amount of outbound rest API calls
Hi,
We are building an application where we need to get the data from the external system required for case processing. To get the data, we need to call two rest API.
1: API -1 gives 1 million or more of records, but we are interested only in one field of these records.
2: API-2 is a call for these 1- Million records separately, i.e. (the same API needs to call for 1 million times ). Again, we are interested in the 1 or 2 fields.
This data can not be cached in pega for a long time. Data get updated in the external system at least close to real-time.
Performance and close to real-time calls are the main factors to consider while designing the solution.
Thanks
BPM Company
NL
You mentioned, that you're interested in only 1-2 records from the whole set.
So why then use API-1? Just make 1-2 separate calls to API-2 to get the exact data what you need
BPM Company
NL
well, it's out of your hands. It's the service owner responsibility to define what exactly will be in the response.
From your side you could add Response Data transform to remove all unneeded properties. But keeping in mind that the connector will retrieve ~1 million records, it will be very heavy =/ Test it beforehand.
Better approach would be to change the webservice to use some kind of pagination from their side