Custom Stream Processing in Data Sets
I found the following on PDN, but the examples are too vague for me to follow. Our requirement is to read ZIP compressed text files that are encoded in a non-unicode format. Pega will unzip for us, but we have to do the string conversion in Custom Java classes
1/ In the input stream processing example they are using INT. Should I use STRING for mine, since that is what I am processing.
This page from 8.2 help shows the data-set rule filled in with custom stream classes:
Which are implemented:
Can anybody help with a better example
Pegasystems Inc.
US
The links are fairly clear to me, it all comes down to the java class pending on your requirement. Are you comfortable with basic java?
Pegasystems Inc.
JP
No problem with the Java - senior engineer in Microsoft for 10 years, plus others.
The problem is that the example does not seem to jive with the usage. The documentation says:
Optional: To provide additional file processing for read and write operations, such as encoding and decoding, define and implement a dedicated interface:
- Select Custom stream processing.
- In the Java class with reader implementation field, enter the fully qualified name of the java class with the logic that you want to apply before parsing.
com.pega.bigdata.dataset.file.repository.streamprocessing.sampleclasses.InputStreamShiftingProcessing
But then the example demonstrates a "int read" function, IE a parsing operation as opposed to an encoding/decoding stream operation BEFORE parsing.
I looked at all of the OOTB data set implementations, but none of them include custom stream processing.
Updated: 9 Sep 2020 0:31 EDT
Pegasystems Inc.
JP
I would expect the solution to be more like the GetEFormFromFile activity, reading the stream into a byte array stream which can then be converted to a Unicode string which I can pass back for pega to parse the contents.
Or are we expected to implement a read operator for every data type that we want to take out of the stream?