Not able to create a single zip File and store in Repository using File Dataset
Hi All,
We have a requirement to create a single zip file from pxDecisionResults DDS and store in the repository.
Created a Data Flow with below configurations:
Source: pxDecisionResults (Output: embedded page-list)
Destination: File Data Set (creating json/zip and storing in Repository)
Current Result: Multiple zip files each for 1 Result.
Example:
pxDecisionResults Output is in below format and has 10 Pages:
pxResults(1) >> pxDecisionResults()
pxModelResults()
.
.
.
.pxResults(10) >> pxDecisionResults()
pxModelResults()
Output: 10 zip files having json respect to each.
Expected Output: 1 Zip file having all the json files
Any help would be needful.
Thanks!
Updated: 21 Feb 2022 6:22 EST
Pegasystems Inc.
NL
@Priyesh - When you export using the File Dataset as a destination shape in the data flow in 8.5, it will create 1 file per partition. This ensures scalability and reliability, the only way to create a single file is if you have a single partition for the entire dataset.
Ernst & Young LLP
IN
@kamanThanks for your response.
I can see the partition key can be provided only when the Source is Report defination. where as in our case, Source is a Data-Set where there is no partition key defined (Hence it should create single zip). Is it taking the primary Keys of Data-Set as Partition keys?
Please correct me if i am wrong.
Thanks!
Pegasystems Inc.
NL
@Priyesh - You can look at the number of partitions created in the Data flow metrics for the Data flow that you run. In this case, since the source of the data flow is pxDecisionResults which is a dataset of type Decision Datastore, the partition key is based on the Key specified in this Dataset. We have different partitioning mechanisms for different datasets and the number of partitions created differs based on the number of nodes and number of threads that you have in your setup.