Question
CIBC
CA
Last activity: 12 Aug 2019 15:29 EDT
com.pega.dsm.dnode.impl.prpc.service.ServiceDeploymentImpl >< BEA-000000 >< Unable to get member information from stale node
Hi All,
We are in the process of upgrading from 7.2.1 to 8.2.2. We have successfully upgraded in the lowest environment and are moving the updated image (whole database image) from lower to higher environments. Our environment is thus now, Pega 8.2.2 on Weblogic 12c and Oracle 11g.
On moving to higher environment(s) we noticed a new recurring warning in our Weblogic (stdout of each node) log almost every minute. A sample of such warning is as given below:
<Aug 7, 2019 4:52:42 PM EDT> <Warning> <com.pega.dsm.dnode.impl.prpc.service.ServiceDeploymentImpl> <BEA-000000> <Unable to get member information from b4f8cab7c9872ff128a82c93223ed2fa and its pulse was last updated 26086852 minutes ago, marking it unreachable>
The node b4f8cab7c9872ff128a82c93223ed2fa is the node from the lowest environment and does not exist in this environment and thus data related to node b4f8cab7c9872ff128a82c93223ed2fa can be considered stale.
We checked pr_sys_statusnodes and it has the records for correct nodes only. We have cleaned the temp directories in each node and have also cleaned any stale data related to the node in the following tables:
PR_DATA_DECISION_DF_MET
PR_HOURLY_USAGE
PR_INDEX_DATA_AGENTS
PR_LOG_CLUSTER
PR_LOG_COLLABORATION
PR_LOG_DATAFLOW_EVENTS
PR_LOG_JOBS_SCHEDULER
PR_LOG_REPORTSTATS
PR_LOG_SYSTEMSTATE
PR_SYSJOBS
PR_PAGE_STORE
PR_SYS_STATUSNODES
PR_SYS_STATUSDETAILS
PR_SYS_UPDATESCACHE
Hi All,
We are in the process of upgrading from 7.2.1 to 8.2.2. We have successfully upgraded in the lowest environment and are moving the updated image (whole database image) from lower to higher environments. Our environment is thus now, Pega 8.2.2 on Weblogic 12c and Oracle 11g.
On moving to higher environment(s) we noticed a new recurring warning in our Weblogic (stdout of each node) log almost every minute. A sample of such warning is as given below:
<Aug 7, 2019 4:52:42 PM EDT> <Warning> <com.pega.dsm.dnode.impl.prpc.service.ServiceDeploymentImpl> <BEA-000000> <Unable to get member information from b4f8cab7c9872ff128a82c93223ed2fa and its pulse was last updated 26086852 minutes ago, marking it unreachable>
The node b4f8cab7c9872ff128a82c93223ed2fa is the node from the lowest environment and does not exist in this environment and thus data related to node b4f8cab7c9872ff128a82c93223ed2fa can be considered stale.
We checked pr_sys_statusnodes and it has the records for correct nodes only. We have cleaned the temp directories in each node and have also cleaned any stale data related to the node in the following tables:
PR_DATA_DECISION_DF_MET
PR_HOURLY_USAGE
PR_INDEX_DATA_AGENTS
PR_LOG_CLUSTER
PR_LOG_COLLABORATION
PR_LOG_DATAFLOW_EVENTS
PR_LOG_JOBS_SCHEDULER
PR_LOG_REPORTSTATS
PR_LOG_SYSTEMSTATE
PR_SYSJOBS
PR_PAGE_STORE
PR_SYS_STATUSNODES
PR_SYS_STATUSDETAILS
PR_SYS_UPDATESCACHE
We are still getting the same warning in the Weblogic log and it seems that Pega has kept the node details somewhere other than the above mentioned tables.
Any help or insight in this matter is highly appreciated.
Please let me know if the issue needs more clarification or if any further detail is required on this.
Thanks,
Tirthendra
Pegasystems Inc.
US
Can you check your decisioning landing page to see if old stale nodes are still there? If yes, decommission those.
see an example from my internal env:
Pegasystems Inc.
US
Thanks for checking! These messages may be coming from Weblogic itself. I suggest you follow the Oracle suggestion to clear the Weblogic temp files/folders.
CIBC
CA
Hi Kevin,
Tried the following steps:
1. Shutdown server
2. Clear Weblogic temp files
3. Re-ran database table data clearing scripts
4. Restarted server
Unfortunately, the warning is still there in the log.
Please let me know if anything else comes to your mind. I am suspecting, there might be blob column(s) that has the stale node details which we are missing as our scripts are checking only for exposed columns.
Thanks,
Tirthendra