Question
JPMC
IN
Last activity: 13 Apr 2022 12:23 EDT
com.pega.pegarules.pub.context.InitializationFailedError: PRNodeImpl init failed
We are seeing the below error when we hit the pega url.
HTTP Status 503 – Service Unavailable.
Pega logs has the following error. Any help is highly appreciated.
com.pega.pegarules.pub.context.InitializationFailedError: PRNodeImpl init failed
We are seeing the below error when we hit the pega url.
HTTP Status 503 – Service Unavailable.
Pega logs has the following error. Any help is highly appreciated.
com.pega.pegarules.pub.context.InitializationFailedError: PRNodeImpl init failed
Exception in thread "StartupTaskUtil INITIALIZE_QUEUE_PROCESSORS" java.lang.StackOverflowError at com.pega.pegarules.exec.internal.util.SpecializerImpl.getInternationalVariants(SpecializerImpl.java:139) at com.pega.pegarules.exec.internal.util.SpecializerImpl.specialize(SpecializerImpl.java:69) at com.pega.pegarules.exec.internal.util.RSLSpecializerImpl.getSpecializedList(RSLSpecializerImpl.java:75) at com.pega.pegarules.exec.internal.basic.RulesetListImpl.applySpecializations(RulesetListImpl.java:1080) at com.pega.pegarules.exec.internal.basic.RulesetListImpl.initialize(RulesetListImpl.java:780) at com.pega.pegarules.exec.internal.basic.ApplicationRulesetListImpl.<init>(ApplicationRulesetListImpl.java:97) at com.pega.pegarules.exec.internal.basic.ApplicationRulesetListImpl.duplicate(ApplicationRulesetListImpl.java:171) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.getReadOnlyList(StaticRulesetListImpl.java:506) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.getReadOnlyList(StaticRulesetListImpl.java:474) at com.pega.pegarules.session.internal.authorization.context.LegacyContext.getRuleSetList(LegacyContext.java:310) at com.pega.pegarules.session.internal.authorization.Authorization.getRuleSets(Authorization.java:669) at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:2625) at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:2595) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.doesRuleSetNameExist(StaticRulesetListImpl.java:404) at com.pega.pegarules.exec.internal.util.SpecializerImpl.doesRuleSetNameExist(SpecializerImpl.java:241) at com.pega.pegarules.exec.internal.util.SpecializerImpl.getInternationalVariants(SpecializerImpl.java:164) at com.pega.pegarules.exec.internal.util.SpecializerImpl.specialize(SpecializerImpl.java:69) at com.pega.pegarules.exec.internal.util.RSLSpecializerImpl.getSpecializedList(RSLSpecializerImpl.java:75) at com.pega.pegarules.exec.internal.basic.RulesetListImpl.applySpecializations(RulesetListImpl.java:1080) at com.pega.pegarules.exec.internal.basic.RulesetListImpl.initialize(RulesetListImpl.java:780) at com.pega.pegarules.exec.internal.basic.ApplicationRulesetListImpl.<init>(ApplicationRulesetListImpl.java:97) at com.pega.pegarules.exec.internal.basic.ApplicationRulesetListImpl.duplicate(ApplicationRulesetListImpl.java:171) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.getReadOnlyList(StaticRulesetListImpl.java:506) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.getReadOnlyList(StaticRulesetListImpl.java:474) at com.pega.pegarules.session.internal.authorization.context.LegacyContext.getRuleSetList(LegacyContext.java:310) at com.pega.pegarules.session.internal.authorization.Authorization.getRuleSets(Authorization.java:669) at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:2625) at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:2595) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.doesRuleSetNameExist(StaticRulesetListImpl.java:404) at com.pega.pegarules.exec.internal.util.SpecializerImpl.doesRuleSetNameExist(SpecializerImpl.java:241) at com.pega.pegarules.exec.internal.util.SpecializerImpl.getInternationalVariants(SpecializerImpl.java:164) at com.pega.pegarules.exec.internal.util.SpecializerImpl.specialize(SpecializerImpl.java:69) at com.pega.pegarules.exec.internal.util.RSLSpecializerImpl.getSpecializedList(RSLSpecializerImpl.java:75) at com.pega.pegarules.exec.internal.basic.RulesetListImpl.applySpecializations(RulesetListImpl.java:1080) at com.pega.pegarules.exec.internal.basic.RulesetListImpl.initialize(RulesetListImpl.java:780) at com.pega.pegarules.exec.internal.basic.ApplicationRulesetListImpl.<init>(ApplicationRulesetListImpl.java:97) at com.pega.pegarules.exec.internal.basic.ApplicationRulesetListImpl.duplicate(ApplicationRulesetListImpl.java:171) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.getReadOnlyList(StaticRulesetListImpl.java:506) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.getReadOnlyList(StaticRulesetListImpl.java:474) at com.pega.pegarules.session.internal.authorization.context.LegacyContext.getRuleSetList(LegacyContext.java:310) at com.pega.pegarules.session.internal.authorization.Authorization.getRuleSets(Authorization.java:669) at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:2625) at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:2595) at com.pega.pegarules.exec.internal.basic.StaticRulesetListImpl.doesRuleSetNameExist(StaticRulesetListImpl.java:404) at com.pega.pegarules.exec.internal.util.SpecializerImpl.doesRuleSetNameExist(SpecializerImpl.java:241)
***Edited by Moderator Marije to add Support Case Details***
Accepted Solution
Updated: 13 Apr 2022 12:23 EDT
Pegasystems Inc.
GB
@BhanuPrakash_GI have checked support ticket INC-214541 and notice that our support team had to close it due to lack of response.
Below the troubleshooting guidelines which were suggested to you.
Our Cassandra team@BhanuPrakash_G notice the following in your previous logs:
Detected powershell execution permissions. @BhanuPrakash_GRunning with enhanced startup scripts.
@BhanuPrakash_G @BhanuPrakash_G WARNING! @BhanuPrakash_GAutomatic page file configuration detected. @BhanuPrakash_G @BhanuPrakash_G It is recommended that you disable swap when running Cassandra @BhanuPrakash_G @BhanuPrakash_G for performance and stability reasons.
@BhanuPrakash_G @BhanuPrakash_G WARNING! Detected a power profile other than High Performance. @BhanuPrakash_G @BhanuPrakash_G Performance of this node will suffer. @BhanuPrakash_G @BhanuPrakash_G Modify conf\cassandra.env.ps1 to suppress this warning.
VerifyPortsAreAvailable : Found a port already in use. Aborting startup
Analysis:
As the DDS runs on@BhanuPrakash_G9042 and the error says the "Found a port already in use. Aborting startup At C:\Program Files\Apache Software Foundation\Tomcat@BhanuPrakash_G9.0_PegaCOE\cassandra\bin\cassandra.ps1:98 char:9".@BhanuPrakash_G
1)@BhanuPrakash_Gcheck if any firewall is enabled that prevents port 9042 from communicating.
2) Check with your network team whether there are connections on this server to port 9042?
@BhanuPrakash_GI have checked support ticket INC-214541 and notice that our support team had to close it due to lack of response.
Below the troubleshooting guidelines which were suggested to you.
Our Cassandra team@BhanuPrakash_G notice the following in your previous logs:
Detected powershell execution permissions. @BhanuPrakash_GRunning with enhanced startup scripts.
@BhanuPrakash_G @BhanuPrakash_G WARNING! @BhanuPrakash_GAutomatic page file configuration detected. @BhanuPrakash_G @BhanuPrakash_G It is recommended that you disable swap when running Cassandra @BhanuPrakash_G @BhanuPrakash_G for performance and stability reasons.
@BhanuPrakash_G @BhanuPrakash_G WARNING! Detected a power profile other than High Performance. @BhanuPrakash_G @BhanuPrakash_G Performance of this node will suffer. @BhanuPrakash_G @BhanuPrakash_G Modify conf\cassandra.env.ps1 to suppress this warning.
VerifyPortsAreAvailable : Found a port already in use. Aborting startup
Analysis:
As the DDS runs on@BhanuPrakash_G9042 and the error says the "Found a port already in use. Aborting startup At C:\Program Files\Apache Software Foundation\Tomcat@BhanuPrakash_G9.0_PegaCOE\cassandra\bin\cassandra.ps1:98 char:9".@BhanuPrakash_G
1)@BhanuPrakash_Gcheck if any firewall is enabled that prevents port 9042 from communicating.
2) Check with your network team whether there are connections on this server to port 9042?
As the port that DDS use is not available you are getting this. you can check "VerifyPortsAreAvailable :
TCP@BhanuPrakash_G@BhanuPrakash_G<removed IP details> SYN_SENT" and 9042.
You were asked to@BhanuPrakash_G verify the below link
https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/configuration/secureFireWall.html
As you are an on-premise client you were asked to check the above comments with your network team.
Note that without the full debug logs the support team were not be able to help further therefore the case is closed.
JPMC
IN
@PoojaGadige Have gone through the articles but none of them have been able to rectify the issue.
Pegasystems Inc.
GB
@BhanuPrakash_G Do you just see errors in the logs, or do users experience application issues?
Can you confirm if this is a new install?
Has there been a DB migration or other similar change?
What is the platform stack you are using?
The error extract you provided includes reference to "INITIALIZE_QUEUE_PROCESSORS".
During system startup "system runtime context(SRC)" will form and loads all the QP rules according to SRC with the given QP configurations where it will read this alert threshold also to raise the alerts appropriately when ever after system startup.
what is the java version used at your end and also JAVA_Home and path variable are set properly?
Can you check the context.xml and prconfig.xml to cross check DB info .
add below to prconfig file with your own cluster name and try:
prconfig/identification/cluster/name.
And PFB the reference for contect.xml configuration:
@BhanuPrakash_G Do you just see errors in the logs, or do users experience application issues?
Can you confirm if this is a new install?
Has there been a DB migration or other similar change?
What is the platform stack you are using?
The error extract you provided includes reference to "INITIALIZE_QUEUE_PROCESSORS".
During system startup "system runtime context(SRC)" will form and loads all the QP rules according to SRC with the given QP configurations where it will read this alert threshold also to raise the alerts appropriately when ever after system startup.
what is the java version used at your end and also JAVA_Home and path variable are set properly?
Can you check the context.xml and prconfig.xml to cross check DB info .
add below to prconfig file with your own cluster name and try:
prconfig/identification/cluster/name.
And PFB the reference for contect.xml configuration:
<Resource name="jdbc/PegaRULES" auth="Container" type="javax.sql.DataSource" maxActive="100" maxIdle="30" maxWait="10000" driverClassName="org.postgresql.Driver" url="jdbc:postgresql://localhost:5432/pega" username="pega" password="pega"/> <Environment name="url/initialization/explicittempdir" value= <Environment name="prconfig/database/databases/PegaRULES/defaultSchema" value= <Environment name="prconfig/database/databases/PegaDATA/defaultSchema" value=
Can you check this post Cannot access pega designer studio after apply changes on prconfig.xml and verify the context.xml entries?
If you need a more in-depth help analyzing your full logs, please can we ask that you log a support incident?
Please provide the INC id here so that we can help you track it.
Updated: 23 Mar 2022 5:56 EDT
JPMC
IN
@MarijeSchillern Thank you for the response. We did a fresh installation of Pega 8.5.2 few weeks ago and everything was work fine (were able to login and do our work) but few days ago when we try to login we are seeing a "HTTP Status 503 – Service Unavailable" and unable to login.
Attaching the full log file for your reference. Appreciate your help.
Thank you.
Pegasystems Inc.
GB
I see Authorization errors.
Also:
Caused by: com.pega.pegarules.pub.PRRuntimeException: Method Invocation exception at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1401) ~[prprivate-session.jar:?] Caused by: java.lang.reflect.InvocationTargetException Caused by: com.pega.pegarules.pub.PRRuntimeException: Startup Task: [INITIALIZE_QUEUE_PROCESSORS] has failed after retries or with error at com.pega.pegarules.session.internal.async.startup.StartupTaskUtil.awaitStartupTasks(StartupTaskUtil.java:145) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1399) ~[prprivate-session.jar:?] ... 57 more Caused by: java.lang.StackOverflowError at com.pega.pegarules.pub.context.ThreadContainer.get(ThreadContainer.java:66) ~[prpublic.jar:?]
I'm wondering if there's a problem with the URL.
Can you please check the DSS settings listed in the following article?
I think it would be best if you engage a support engineer to go into a more in-depth help analysis of your logs.
Please can we ask that you log a support incident?
Please provide the INC id here so that we can help you track it.
JPMC
IN
@MarijeSchillern Thank you for your time in responding to my question. As suggested have opened
| INC-214541 |
Pegasystems Inc.
GB
@BhanuPrakash_G many thanks, I've added the ticket id to this post and we will track this in parallel to allow other users to benefit once the issue is identified by our support team.
Accepted Solution
Updated: 13 Apr 2022 12:23 EDT
Pegasystems Inc.
GB
@BhanuPrakash_GI have checked support ticket INC-214541 and notice that our support team had to close it due to lack of response.
Below the troubleshooting guidelines which were suggested to you.
Our Cassandra team@BhanuPrakash_G notice the following in your previous logs:
Detected powershell execution permissions. @BhanuPrakash_GRunning with enhanced startup scripts.
@BhanuPrakash_G @BhanuPrakash_G WARNING! @BhanuPrakash_GAutomatic page file configuration detected. @BhanuPrakash_G @BhanuPrakash_G It is recommended that you disable swap when running Cassandra @BhanuPrakash_G @BhanuPrakash_G for performance and stability reasons.
@BhanuPrakash_G @BhanuPrakash_G WARNING! Detected a power profile other than High Performance. @BhanuPrakash_G @BhanuPrakash_G Performance of this node will suffer. @BhanuPrakash_G @BhanuPrakash_G Modify conf\cassandra.env.ps1 to suppress this warning.
VerifyPortsAreAvailable : Found a port already in use. Aborting startup
Analysis:
As the DDS runs on@BhanuPrakash_G9042 and the error says the "Found a port already in use. Aborting startup At C:\Program Files\Apache Software Foundation\Tomcat@BhanuPrakash_G9.0_PegaCOE\cassandra\bin\cassandra.ps1:98 char:9".@BhanuPrakash_G
1)@BhanuPrakash_Gcheck if any firewall is enabled that prevents port 9042 from communicating.
2) Check with your network team whether there are connections on this server to port 9042?
@BhanuPrakash_GI have checked support ticket INC-214541 and notice that our support team had to close it due to lack of response.
Below the troubleshooting guidelines which were suggested to you.
Our Cassandra team@BhanuPrakash_G notice the following in your previous logs:
Detected powershell execution permissions. @BhanuPrakash_GRunning with enhanced startup scripts.
@BhanuPrakash_G @BhanuPrakash_G WARNING! @BhanuPrakash_GAutomatic page file configuration detected. @BhanuPrakash_G @BhanuPrakash_G It is recommended that you disable swap when running Cassandra @BhanuPrakash_G @BhanuPrakash_G for performance and stability reasons.
@BhanuPrakash_G @BhanuPrakash_G WARNING! Detected a power profile other than High Performance. @BhanuPrakash_G @BhanuPrakash_G Performance of this node will suffer. @BhanuPrakash_G @BhanuPrakash_G Modify conf\cassandra.env.ps1 to suppress this warning.
VerifyPortsAreAvailable : Found a port already in use. Aborting startup
Analysis:
As the DDS runs on@BhanuPrakash_G9042 and the error says the "Found a port already in use. Aborting startup At C:\Program Files\Apache Software Foundation\Tomcat@BhanuPrakash_G9.0_PegaCOE\cassandra\bin\cassandra.ps1:98 char:9".@BhanuPrakash_G
1)@BhanuPrakash_Gcheck if any firewall is enabled that prevents port 9042 from communicating.
2) Check with your network team whether there are connections on this server to port 9042?
As the port that DDS use is not available you are getting this. you can check "VerifyPortsAreAvailable :
TCP@BhanuPrakash_G@BhanuPrakash_G<removed IP details> SYN_SENT" and 9042.
You were asked to@BhanuPrakash_G verify the below link
https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/configuration/secureFireWall.html
As you are an on-premise client you were asked to check the above comments with your network team.
Note that without the full debug logs the support team were not be able to help further therefore the case is closed.