API for streaming bytes for the output response
Java servlet developers are used to using ServletOutputStream to write bytes directly to the output stream.
Is this possible in Pega? (I thought this was added some time back, but I can't find it now).
Through v7.1.5, as I'm aware the sendFile() API reads a PRFile or a byte array -- which it then writes into the ServiceExport path, which isn't very efficient.
We need a more efficient way of reading a stream from a URL and then writing to the output stream.
Jon
***Updated by moderator: Lochan to close post***
This post has been archived for educational purposes. Contents and links will no longer be updated. If you have the same/similar question, please write a new post.
Pegasystems Inc.
CA
Hey Jon,
Can you explain your use case a bit better? It sounds like a simple AJAX scenario (read a stream from a URL, write it to the screen).
-Ben
BNY Mellon
US
Ben - nothing to do with Ajax. It's the server that needs to read from the URL.
The server is reading from an ECM system via HTTP. That's easy enough to do; in traditional Java servlet programming one would pipe it into ServletOutputStream to run efficiently. But that's not directly accessible in Pega.
Pegasystems Inc.
GB
I am not sure about this - but I'll post the idea anyway for discussion:
Could you create a custom PRPC CONTROL (or just a Rule-File-HTML?) which does the reading of the remote system (via a buffer I guess) and outputs (via standard JSP /embedded "<%...%>" Java code) and then do a 'Show-Stream' on this control / Rule-File-HTML ?
Would this result in a temporary file being created in StaticContent - or would everything get done in memory ?
BNY Mellon
US
Not sure what that would accomplish -- the existing JSP tags assume that the data is all there, they don't support streaming.
Also, since we're serving a binary file, that is served straight via HTTP, and not via an HTML container.
Updated: 16 Jul 2015 11:19 EDT
Pegasystems Inc.
GB
So what I meant was something like this : we aren't really writing a JSP - just using it (probably very wrongly) to create a Servlet to Stream Bytes.
My testing so far suggests this is a probably a very bad idea (or I have messed-up my code - or both) - I'm getting OOM errors using this.
I create a 'Rule-Obj-HTML' like this:
<%
int filesize=1024 * 1024 * 100;
int flush_point=1024 *32;
ClipboardPage objHeadersPage = tools.findPage("pxRequestor").getProperty(".pyHTTPResponseHeaders").getPageValue();
objHeadersPage.putString("contentType" ,"application/octet-stream");
objHeadersPage.putString("ContentDisposition" ,"attachment;filename=binary.dmp");
OutputStream os = mStreamResponse.getOutputStream();
BufferedOutputStream bos=new BufferedOutputStream( os );
for (int b=0;b<filesize;b++) {
bos.write(0);
if ( (b % flush_point)==0) {
oLog.infoForced("Flushing buffer");
bos.flush();
}
}
bos.flush();
bos.close();
%>
And then use a 'Show-Stream' (note, 'preview' will 'decorate' the content with HTML/JS/CSS so that it is no good). from an activity.
This works for smaller (<10Mb) on my system - but blows up badly over that....
So what I meant was something like this : we aren't really writing a JSP - just using it (probably very wrongly) to create a Servlet to Stream Bytes.
My testing so far suggests this is a probably a very bad idea (or I have messed-up my code - or both) - I'm getting OOM errors using this.
I create a 'Rule-Obj-HTML' like this:
<%
int filesize=1024 * 1024 * 100;
int flush_point=1024 *32;
ClipboardPage objHeadersPage = tools.findPage("pxRequestor").getProperty(".pyHTTPResponseHeaders").getPageValue();
objHeadersPage.putString("contentType" ,"application/octet-stream");
objHeadersPage.putString("ContentDisposition" ,"attachment;filename=binary.dmp");
OutputStream os = mStreamResponse.getOutputStream();
BufferedOutputStream bos=new BufferedOutputStream( os );
for (int b=0;b<filesize;b++) {
bos.write(0);
if ( (b % flush_point)==0) {
oLog.infoForced("Flushing buffer");
bos.flush();
}
}
bos.flush();
bos.close();
%>
And then use a 'Show-Stream' (note, 'preview' will 'decorate' the content with HTML/JS/CSS so that it is no good). from an activity.
This works for smaller (<10Mb) on my system - but blows up badly over that....
(EDIT: I had the wrong version of the code there - should be writing a zero byte everytime as padding - rather than an the value of 'b' which would go above the byte-boundary and change the expected filesize)
EDIT 2 : Had the wrong rule type in there (should be "Rule-Obj-HTML", was "Rule-File-HTM").
BNY Mellon
US
What's a Rule-File-HTM? You mean Rule-Obj-HTML?
I've not previously seen mStreamResponse documented, but it looks like a protected property in StreamBuilderBase, of type JSPResponse (which is an extension of HttpServletResponse, which is what I was looking for).
So this could be the HttpServletResponse object we were looking for.
Pegasystems Inc.
GB
Sorry - I had the wrong rule-type in there - have corrected it. (you are right : It was a "Rule-Obj-HTML").
I stole the 'mStreamResponse' by doing a 'view java' on the Rule-Obj-HTML file - and guessed it would probably be the HttpServletResponse - but I don't know what behaviour it has in addition to the original object.
So - what I did probably counts as a hack (especially considering it really did cause problems when I tested with it) - and unfortunately I think that means you shouldn't rely on it....
What about the alternative 'sendFile' method signature ? (Where you can supress the write-to-file behaviour ?)
BNY Mellon
US
Because it's not a stream, which could leverage buffering. It copies all of the characters into memory at once before shipping off to the client.
BNY Mellon
US
Some months later... tested this out. This works for text content, but not quite for binaries -- the bytes sent back to the client are not the ones I expect.
Any clues?
Pegasystems Inc.
GB
Or as Obi-Wan Kenobi might have put it "These aren't the bytes you are looking for....."
Do you have an example of what was returned ? Are they base64-encoded bytes; or a buffer which is 'overfilled' etc ?
BNY Mellon
US
I couldn't really see a pattern. It did not look like the Base64 equivalent (as I would have seen a sequence of characters). I can try outputting this again to share here.
Pegasystems Inc.
GB
Hi Jon,
I haven't tried this yet - but going back to the 'sendFile()' API - it looks like you can switch off the behaviour where the file-data is cached on the Server Folder ?
java.lang.String sendFile(byte[] aFileData,
java.lang.String aFileName,
boolean aPersistFileToServiceExport,
StringMap aHttpHeaders,
boolean aSendForDownload)
Send file data in aFileData.
Parameters:
aFileData - file data to be sent to client.
aFileName - file name to use when sending or writing to server's ServiceExport directory the file data.
aPersistFileToServiceExport - should this file be written to server's ServiceExport directory. set this to true only if the file will be accessed multiple times.
aHttpHeaders - specifies custom http headers to use when sending this file data (overrides aSendForDownload).
aSendForDownload - should this file be sent for download (if true, appropriate HTTP headers get added).
Returns:
null if successful, otherwise check returned string for error messages.
Does this do what you need to do here ?
John