Hazlecast: Member has left the cluster
Hi All,
We are running the NBAD scheduler campaign, and it has failed with the error:
2022-12-27 13:48:40,850 [ PegaRULES-Batch-15] [ STANDARD] [ ] [ BCARL:01.02.01] (PegaMKT_Work_ProgramRun.Action) ERROR BE2N8FRDZLWVNVBASIN39LX6RSDVPJ9XRA - Failure in ExecuteProgramRun: PRRuntimeError:PR-70014
and we found the following error has been observed in the log:
2022-12-27 12:05:01,632 [d0e90dc72a30.event-3] [ ] [ ] [ ] (til.HazelcastMembershipManager) INFO - Member has left the cluster: Member: [name=eb06fdafab8102eb1131ab3b508317aa, address=RHU3PEGADTFL01/ Proprietary information hidden:5701, uuid=d787d1b5-e60a-462e-a72a-37672dd0cdba, member version=3.12.10, mode=SERVER]
Could someone please provide insights about this? Thank you.
PFA's for the logs
Accepted Solution
Updated: 2 Mar 2023 4:56 EST
LTIMindtree
SA
Member left the cluster issue could be for multiple reasons.
1. In-sufficient Requestor Pool Thread
2. Hung threads
3. If Stream, the member is trying to refer the old kafka partitions which requires cleaning of System cache Tables etc..
If your application is running in On-Prem, You can idenitfy the node type and perform the rollover restart to monitor if it's joining the cluster back or not. Sometimes, it's required to perform the whole system maintenance if Stream node left the cluster or any issues with Cluster on Joining back.
It's advisable to raise an INC in MSP portal for further alaysis on your application.
Anabatic Technologies Tbk Pt
ID
Hi @Kishore Sanagapalli Thank you for your response. Our application is running on-premises, and for this issue, we performed the restart and monitored the cluster joining back, and our campaign is running properly again. Thank you.