JMS Listener error handling on database unavailability
Hi Pega Developers,
When our database is not available (due to some maintanance tasks) our JMS Listener is not able to process messages. Every consumed message ends with error which is send to reponse destination (queue) configured on JMS properties tab.
Error message contains only:
JMS service [PPEJMS][ServiceJMS][createProcess] failed:Service requestor could not be authenticated
Is there any way to keep original message which was not processed to be requeued? Maybe Pega has some DSS which cause to save original message in destination queue along with error message?
Proximus
BE
Hello,
if it's a planned activity, I would mainly advise to stop your application, which would then lead to have your listeners down, so not consumed, so no error to handle.
Knowing the DB is down (planned), there's no way you can handle & process any messages.
Regards
Anthony
Updated: 17 Jul 2023 9:38 EDT
1985
PL
@Anthony_GourtayThanks for reply. What if we face DB unplanned failure? How to avoid lost messages?
Proximus
BE
If there's unexpected downtime then your listener will not be able to commit the transaction and this one will be rolled back, ie, message will be back to external queue.
This said, when working with PEGA applications you clearly need to have a strong DB infra with at least a hot fail over which will "immediately" bring up a new DB instance if your main server is down. And monitoring of course which will tell that DB is down and urgent action is needed.
With such basics, your DB will never be down for long or at least you'll quickly be able to stop your application while fixing the DB issue. There's no point to keep your application servers up if DB is known down and not soon up. You'll be on safer side to make a clean restart afterwards anyway
Regards
Anthony
1985
PL
@Anthony_Gourtay Thank you for your help.
You wrote that during DB downtime message is sent back to external queue. By external queue you mean the one which listener is listen on? or the queue used by listener to send responses?
As I wrote in our case we received information that listener had problems (message sent to response queue) but it contains only exception details, without original payload.
Is it possible to somehow configure error messages sent to response queue, they would contain original payload?
Proximus
BE
I understood that it was a regular JMS service so I guessed that your application would listen to an external queue messaging system.
Logically for the communication with this external queuing, the original message shouldn't be deleted if you don't acknowledge that you handled it. Otherwise, you would easily loose messages for any reason and you'll be blind in where's the issue.
You want to make things tricky with your listener handling while your issue is with the DB.
So put focus on properly & quickly handling your DB issues and you'll have less other single exception to handle.
Then for your listener processing, this depends on your business and the way you implemented your exception handling.
There are clearly lots of things which can be done.
But still, if you're not able to immediately see that your DB is down, then this is where you need to put focus.
Regards
Anthony