Question
Ford Motor Private Ltd
IN
Last activity: 7 Oct 2019 1:45 EDT
Unable to combine two data sets in data flow
Hi All,
We have a requirement where in bulk data from multiple data source will come. We have tried to create a data set uploading a data from csv file. When i tried to combine 2 different data sets using compose for the second data set i am getting error "
I have defined primary key as ID for both first and second data set . I am unable to figure out the way to fix it. Any inputs or help is appreciable.
Thanks in advance,
Ravikiran C
Pegasystems Inc.
US
I think that is expected as your secondary dataset does not support 'browse by keys', see this link:
Ford Motor Private Ltd
IN
Thank you Kevin for sharing your thoughts. We have decided not to go for Data flows for our requirement as we cant acheive it completly.
Pegasystems Inc.
NL
The CSV dataset indeed doesnt support that, what it really means that there is no way of indexing the records in such a dataset quickly. The workaround is to copy the CSV dataset into something that can be used for a join/merge, most commonly a DDS (Cassandra) dataset. So you first run a dataflow that copies the entire CSV into a DDS set (with appropriate indices defined), then join/merge that one in your main flow.
Ford Motor Private Ltd
IN
Thank you Otto for sharing your thoughts. We have decided not to go for Data flows for our requirement as we cant acheive it completly.