Question
Pegasystems Inc.
US
Last activity: 17 May 2016 6:15 EDT
Email Listener poor performance on Pega 7.1.7 - Need design recommendations
Hi All,
Posting this on behalf of client as they need some design recommendations to improve the EmailListener performance.
Have 1 Email Listener instance running on one node to process incoming email messages, the listener service activity does the following as soon as an email gets picked up.
Create a new work object, Create PDF from email content, Attach PDF, and call some external REST service to upload document content. These are the whole bunch of operations which runs on EmailListener thread for a single email. This usually takes 2-3 mins to finish the job depends on the size of the email content.
On an average, this listener has to process 200-250 emails per day and have the below configurations set on Email Listener.
With the provided details on the load, processing time, configuration -
1. User would like to know if there are any recommendations to improve the listener performance? (like increasing concurrent threads or have another listener thread instance?)
Hi All,
Posting this on behalf of client as they need some design recommendations to improve the EmailListener performance.
Have 1 Email Listener instance running on one node to process incoming email messages, the listener service activity does the following as soon as an email gets picked up.
Create a new work object, Create PDF from email content, Attach PDF, and call some external REST service to upload document content. These are the whole bunch of operations which runs on EmailListener thread for a single email. This usually takes 2-3 mins to finish the job depends on the size of the email content.
On an average, this listener has to process 200-250 emails per day and have the below configurations set on Email Listener.
With the provided details on the load, processing time, configuration -
1. User would like to know if there are any recommendations to improve the listener performance? (like increasing concurrent threads or have another listener thread instance?)
2. User requires an API to check the listener status and the last execution date time for monitoring? (suggested to use AES or write a custom java to access the MBean)
Any observations/suggestions would be a great help.
Accepted Solution
ANZ Banking Group
AU
This could definitely be designed better by splitting up the operations in to different transactions. If all of the operations are performed in the single transaction,there could be multiple failure points due to which the entire transaction could fail. For example, If not handled properly, if a there is a failure in the rest call,or pdf generation failure, the creation of the work object may fail which in turn may rollback the entire transaction. Also, there will be obvious performance issues. 200 to 250 emails per day is not a huge load but from the fail over perspective you can enable the email listener on one more node.
Coming to the performance part, 2 to 3 minutes is a lot and this can be handled in a better way.
As Jeff mentioned, If they have the infrastructure to do so, have the email listener run on a separate background node where there will be no users. This way users will not be affected with the performance of the background activities.
Another important thing to do is, Have the email listener just create the work object and schedule the work object to an agent to perform the rest of the operations. This way the work object is committed as soon as it is created and the listener will be done with its processing on the current item and pick up the next item for processing. Rest of the operations like creating PDF, Attaching the PDF and the rest call to store the document can be done by the agent. This way you are splitting the single transaction in to multiple transactions.
Pegasystems Inc.
US
- If part of the email processing is to create a PDF, there is an inherent performance drop because PDF creation is a complicated process. To verify, the customer should disable the PDF creation step of the process and check numbers afterwards. If they have the infrastructure to do so, they may want to run the email listener on a separate node, as the processing performance will be degraded by other load on the system.
- I do not have any suggestions for the API that checks the listener status. Perhaps you can add other categories (is System Management an option?) so that someone else can weigh in.
For the question #2, please refer - JMX API code sample for PRPC MBeans
Accepted Solution
ANZ Banking Group
AU
This could definitely be designed better by splitting up the operations in to different transactions. If all of the operations are performed in the single transaction,there could be multiple failure points due to which the entire transaction could fail. For example, If not handled properly, if a there is a failure in the rest call,or pdf generation failure, the creation of the work object may fail which in turn may rollback the entire transaction. Also, there will be obvious performance issues. 200 to 250 emails per day is not a huge load but from the fail over perspective you can enable the email listener on one more node.
Coming to the performance part, 2 to 3 minutes is a lot and this can be handled in a better way.
As Jeff mentioned, If they have the infrastructure to do so, have the email listener run on a separate background node where there will be no users. This way users will not be affected with the performance of the background activities.
Another important thing to do is, Have the email listener just create the work object and schedule the work object to an agent to perform the rest of the operations. This way the work object is committed as soon as it is created and the listener will be done with its processing on the current item and pick up the next item for processing. Rest of the operations like creating PDF, Attaching the PDF and the rest call to store the document can be done by the agent. This way you are splitting the single transaction in to multiple transactions.
Updated: 7 May 2016 14:46 EDT
Pegasystems Inc.
US
Very good point about having the listener schedule the work so that the listener thread may return to its duties!