While reading a file using a data set we can set the file path something like this "/main/payments/*_Transaction_%{yyyyMMdd_HHmm}.csv". This gives some ability for dynamically calling different files of same type.

However, the problem occurs when you have more than one file of same type lets say something like this 01_Transaction_20211215_1406.csv and 02_Transaction_20211215_1406.csv. In this scenario there is no way to control the file that we want to read.

When a file is being read from a file listener, the processed file is automatically moved/deleted. The same does not happen when the file is being processed from a data flow following this design - https://docs-previous.pega.com/pega-customer-decision-hub-implementation-guide/86/ingesting-customer-data-pega-customer-decision-hub-pega-cloud

Here some java has to be written in order to move the file to a different location (something for which I would want Pega to provide api like they have given for creating a  new file, deleting file etc. D_pxNewFile, D_pxDeleteFile). In case the java for some reason is unable to move the file, we may have a scenario were we have multiple files of same type and data flow might start processing a file which was not intended or which has already been tried.

IDEA - Parametrize or use property reference to pass file path in a file type data set