How to output JSON file via file dataset instead of newline-delimited JSON array?
Hi, Pega Support
We noticed when we output Decision strategy result via dataflow to a file dataset in JSON format, it only create NDJSON (newline-delimited JSON array) format not true JSON file as downstream process expected. Instead of write custom activity or java function to loop thru those NDJSON array and append ; at each line and open/close [ ] bracket and write to file, any better way or OOTb to handle this? We don't want to create long pagelist to convert especially there will be close to 2 million record entries in this output file. Any suggestions and recommendation?
Thanks!
@SHIZ1 Pega’s File Dataset in JSON format outputs data in NDJSON (Newline-Delimited JSON) by default, where each line represents a separate JSON object. Unfortunately, Pega does not have an out-of-the-box (OOTB) feature to directly write a standard JSON array (i.e., a single JSON file with records wrapped in
[ ]brackets and comma-separated). To avoid creating a huge page list or writing a custom activity for post-processing, one workaround is to use an external lightweight ETL tool or script (like a simple Python script or Unix tool likejq) to wrap the NDJSON output into a JSON array. This avoids the performance and memory overhead in Pega while keeping the process simple and scalable. Another option is to write the file as NDJSON and add metadata in the file header to inform the downstream system to treat each line as a record, if that’s acceptable for integration.