Our application generates a lot of data that is usually not tied to case data and currently getting stored in separate tables in the PegaDATA database.
This approach was acceptable till we were handling low volume; with volume increase, we are seeing two issues
1: Pega cloud DB Space
2: Query performance through report definition is getting degraded.
We are thinking of keeping this data outside the Pega database and putting it into Amazon RDS. Use those tables as external tables. We want to understand if it is terrible because record creation is happening in pega, and the daily volume is at least 1 million.
With this approach, data will be outside, and if we want to perform any read operation, we can achieve it on a limited dataset (24hrs)
The external database can perform data separation 24 hrs Vs. the whole dataset.
We don't have to run a purge utility in pega to delete any rows.
Please lets us if you have experience with implementing similar use case or do you see any issues with the idea of