Question
GB
Last activity: 14 Jul 2023 4:57 EDT
Can the DX API be used to extract data from a pega database?
I would like to set up a process for extracting data from a pega cloud database into another data warehouse for reporting.
Preferably, I would like to write a python script which automates the process by connecting with an API.
Can the Pega DX API be used for this type of database extraction process?
Accepted Solution
Updated: 14 Jul 2023 4:43 EDT
Masking Technology
NL
Theoretically you could, but I doubt this is what you want to do.
If you really want to use the Pega API, it allows you to call a data page. This data page would need to do the heavy lifting of gathering the information you want to send to a reporting database. Be aware that this data page runs on the same nodes as the users and the node is not configured to run these heavy processes. It would bother them.
For extracting data and moving it to a data warehouse, BIX is the OOTB functionality for this. Just dive into BIX and read up on the possibilities.
I recently created a BIX implementation that uses a file listener to listen for the BIX export and after it reads the file, it extracts the contents we need and send it to another location using a rest connector. It's a bit like the FTP example pega already provides as a reference.
Updated: 14 Jul 2023 4:57 EDT
GB
@basmasking Thank you, that is really useful! I think I will need to do a bit more reading up on how this would work end to end.
Are you aware of any online resources/tutorials for configuring the BIX process to ingest the data into a data warehouse? (e.g. bigquery, redshift, ...)