Question
DXC Technology
BE
Last activity: 7 Feb 2018 3:00 EST
Nodes recover upon failure causing listeners to fail on it
Random Node of the cluster (upon any type of crash) try to recover but have the following issue after restart :
[.PRPCWorkManager : 2] [ STANDARD] [ MCOMSuper:01.42] (nector.Rule_Connect_JMS.Action) ERROR Rule-Connect-JMS.CCC-Int-Notify.SendNotificationMsgAgent - Caught unhandled exception: java.lang.UnsupportedOperationException: DEST_D_NULL< java.lang.UnsupportedOperationException: DEST_D_NULL
The above error causes that particular JMS-Connect to fail. Thus any messages that were supposed to be produced by it also.
We store the connection details in a declare page which if it fails to load during start up causes issues.
How do we go about preventing this during start up. ? Is there a way to fail the whole node if there were such failures to load during start up ?
Pegasystems Inc.
US
JMS Listeners can throw Exceptions because the Application server starts the listener threads before the PegaRULES engine has fully initialized.
If there are existing queue items in the physical JMS/MQ destinations (queues) and you have configured PRPC Listeners to connect/listen to these destinations then performing the following approach will help resolve the JMS/MQ listener errors on startup (before PegaRULES is initialized).
1. Add the following Dynamic System Settings, Save the DSS –
Short Description: Disable JMS Listeners on startup
Owning Ruleset: Pega-Engine
Purpose: prconfig/initservices/initjms/default
Value: false
Short Description: Disable MQ Listeners on startup
Owning Ruleset: Pega-Engine
Purpose: prconfig/initservices/initmq/default
Value: false
2. Restart the server.
3. After the PegaRULES engine is successfully started, navigate to System Management Application > Node > Listener Management > Select the specific listener and click “Start”.
DXC Technology
BE
Thanks for your input.
But this is not the kind of solution i am looking for as the problem is random. In case of production its a never.
Either of the two below is what i am looking at :
a) I am expecting the servers to do their routine and when a problem like this arises, i am looking for a mechanism to catch it during the start up itself so that it doesnt cascade down to the messages.
b)When the server starts up with this connection issue, the messages which fail due to connection issues fail without retry (even if retry is configured), needing the help of operations to retrigger them manually after we identify the faulty node and do a kill and restart. I am expecting such messages in the queue of the node having connection issues to be redirected to a healthy node instead and retried there rather than letting the situation get out of hand and present a large number of message pile up with errors for operations to work on. In production, this is a dilemma.
thanks,
Jagan
Pegasystems Inc.
US
Hi Jagan,
This needs to be investigated and researched in detail. So please create a PegaSupport ticket so that our Engineers can assist you towards a resolution.
Thanks,
Ankur
Pegasystems Inc.
IN
Hi Jagan,
Please let us know the SR ID back here once you have had the chance to create it. We will tag it to your post above and continue to track it until resolution.
Regards,