Question
JP Morgan Chase
US
Last activity: 7 Jul 2023 13:21 EDT
File Listener Processing error
File listener is not processing if the file content more than 500 records in csv file.
***Edited by Moderator Marije to add Support Case Details***
Accepted Solution
Updated: 7 Jul 2023 13:21 EDT
Pegasystems Inc.
GB
@RAMAKRISHNA JAGU I can see that you logged INC-A1285 at the same time you logged this PSC question.
I believe our GCS team provided you with the following analysis on June 13th:
*** PLEASE MARK THIS REPLY WITH ACCEPT SOLUTION IF YOUR ISSUE IS NOW RESOLVED **
Investigation carried out: Enabled file listener debug loggers from admin studio and stared file listener on one web node. Placed a file with 500 records and started tracing the file listener. Tracing stopped during the processing with error 'unable to reach remote server'. Observed that file listener processing failed and the web node went down. Below are the errors in the logs:
com.pega.pegarules.pub.PRRuntimeException: Incorrect usage - Cannot change state of Auto Populate when Clipboard Declarative Processing is disabled
com.pega.pegarules.pub.context.StaleThreadError: Thread has been recycled
com.pega.pegarules.integration.engine.internal.services.file.FileListener|Exception caught while processing service files.|| com.pega.pegarules.pub.context.StaleRequestorError: PRRequestorImpl was explicitly destroyed
On the service package of the file listener, "Pooling" tab is not available.
@RAMAKRISHNA JAGU I can see that you logged INC-A1285 at the same time you logged this PSC question.
I believe our GCS team provided you with the following analysis on June 13th:
*** PLEASE MARK THIS REPLY WITH ACCEPT SOLUTION IF YOUR ISSUE IS NOW RESOLVED **
Investigation carried out: Enabled file listener debug loggers from admin studio and stared file listener on one web node. Placed a file with 500 records and started tracing the file listener. Tracing stopped during the processing with error 'unable to reach remote server'. Observed that file listener processing failed and the web node went down. Below are the errors in the logs:
com.pega.pegarules.pub.PRRuntimeException: Incorrect usage - Cannot change state of Auto Populate when Clipboard Declarative Processing is disabled
com.pega.pegarules.pub.context.StaleThreadError: Thread has been recycled
com.pega.pegarules.integration.engine.internal.services.file.FileListener|Exception caught while processing service files.|| com.pega.pegarules.pub.context.StaleRequestorError: PRRequestorImpl was explicitly destroyed
On the service package of the file listener, "Pooling" tab is not available.
Conclusion: Seen that the JVM argument "-DNodeSettings=Pega-IntegrationEngine/EnableRequestorPools" value is set to 'false'. Due to this, 'Pooling' tab is not available in service package.
Suggest:
Please update this value to "true" to enable the "Pooling" tab.
Once it is available, increase the "Maximum active requestors" value to '20' and test the file listener processing.
Also it is recommended to enable file listener to run only on 'BackgroundProcessing' nodes.
In file listener -> in Properties tab -> select "Startup option" as "NodeClassification based startup" -> select "Node Type" as "BackgroundProcessing".
Next Actions: Our Support team is waiting for your update to confirm it is resolved.
Pegasystems Inc.
GB
From pega side there is no restriction as such to limit the processing of record. therefore you can process unlimited records.
Could you check exactly when this error occurs by running a Tracer ? This could be because of the activity performed by the thread takes more time to complete/timed out.
Also see InterruptedException: Requestor has been destroyed or recycled
Note: File Listener API no longer reads the entire file into memory. But it streams in the file.So there is no file size limit except for space on the server.
Can you replicate the issue with any simple File Listener configuration and file?
The error seems to be "Thread has been recycled "
See High-throughput file processing for tips on how to prevent long-running requestors.
If you are unable to debug and check your troubleshooting steps, then please log a support incident via the MSP and let us know the INC number here so that we can help track it.
Accepted Solution
Updated: 7 Jul 2023 13:21 EDT
Pegasystems Inc.
GB
@RAMAKRISHNA JAGU I can see that you logged INC-A1285 at the same time you logged this PSC question.
I believe our GCS team provided you with the following analysis on June 13th:
*** PLEASE MARK THIS REPLY WITH ACCEPT SOLUTION IF YOUR ISSUE IS NOW RESOLVED **
Investigation carried out: Enabled file listener debug loggers from admin studio and stared file listener on one web node. Placed a file with 500 records and started tracing the file listener. Tracing stopped during the processing with error 'unable to reach remote server'. Observed that file listener processing failed and the web node went down. Below are the errors in the logs:
com.pega.pegarules.pub.PRRuntimeException: Incorrect usage - Cannot change state of Auto Populate when Clipboard Declarative Processing is disabled
com.pega.pegarules.pub.context.StaleThreadError: Thread has been recycled
com.pega.pegarules.integration.engine.internal.services.file.FileListener|Exception caught while processing service files.|| com.pega.pegarules.pub.context.StaleRequestorError: PRRequestorImpl was explicitly destroyed
On the service package of the file listener, "Pooling" tab is not available.
@RAMAKRISHNA JAGU I can see that you logged INC-A1285 at the same time you logged this PSC question.
I believe our GCS team provided you with the following analysis on June 13th:
*** PLEASE MARK THIS REPLY WITH ACCEPT SOLUTION IF YOUR ISSUE IS NOW RESOLVED **
Investigation carried out: Enabled file listener debug loggers from admin studio and stared file listener on one web node. Placed a file with 500 records and started tracing the file listener. Tracing stopped during the processing with error 'unable to reach remote server'. Observed that file listener processing failed and the web node went down. Below are the errors in the logs:
com.pega.pegarules.pub.PRRuntimeException: Incorrect usage - Cannot change state of Auto Populate when Clipboard Declarative Processing is disabled
com.pega.pegarules.pub.context.StaleThreadError: Thread has been recycled
com.pega.pegarules.integration.engine.internal.services.file.FileListener|Exception caught while processing service files.|| com.pega.pegarules.pub.context.StaleRequestorError: PRRequestorImpl was explicitly destroyed
On the service package of the file listener, "Pooling" tab is not available.
Conclusion: Seen that the JVM argument "-DNodeSettings=Pega-IntegrationEngine/EnableRequestorPools" value is set to 'false'. Due to this, 'Pooling' tab is not available in service package.
Suggest:
Please update this value to "true" to enable the "Pooling" tab.
Once it is available, increase the "Maximum active requestors" value to '20' and test the file listener processing.
Also it is recommended to enable file listener to run only on 'BackgroundProcessing' nodes.
In file listener -> in Properties tab -> select "Startup option" as "NodeClassification based startup" -> select "Node Type" as "BackgroundProcessing".
Next Actions: Our Support team is waiting for your update to confirm it is resolved.