Question
Instellars Global Consulting Pvt. Ltd
IN
Last activity: 15 Dec 2022 8:43 EST
Probing Issues Post the Platform Upgrade
Issue Description:
During the any outages (App server or DB or Infra Related ones) while the Application was UP and Running, The Probing Issues occur usually in the Kubernetes Enviornment. During such scenarios, Application will not be available eventhough Infrastructure Outage resolved and try to Bring up the Application.
***Edited by Moderator Marije to add Support Case Details INC-230280 ***
Updated: 15 Dec 2022 8:43 EST
Instellars Global Consulting Pvt. Ltd
IN
So Far: In-order to fix the Probing issues, Amending the Startup probing parameters in the Helm Deployment file / Values file is of no use. We have tried multiple times. In-fact, Increased the Startup Probe values to 6Hours and tried but did not work.
Applied Work Arround:
In-order to fix the issue post our Infrastructure outage, We did re-run the same version of Pega upgrade.sh / upgrade.bat Pega Script over the existing Schemas and re-deployed the Application worked actually. E.g. We were using the 8.6.3 Platform and Same version of Pega Upgrade Script has been invoked and post the Script Completion, Probing Issues were resolved and the Application was back online with No changes in the Probing Values adjusted.
Current Status:
INC-225663 & INC-230280 raised with Pega GCS to find out the Root Cause. Permanent Resolution will be posted back here once it's concluded.
Note: Before applying the Work Arroud We identified to fix the issue, Make sure you take the necessary back up of DB and respective Deployment and config files. I welcome comments and solutions from other readers over the post about the issue and solution.
Accepted Solution
Updated: 15 Dec 2022 8:43 EST
Instellars Global Consulting Pvt. Ltd
IN
So Far: In-order to fix the Probing issues, Amending the Startup probing parameters in the Helm Deployment file / Values file is of no use. We have tried multiple times. In-fact, Increased the Startup Probe values to 6Hours and tried but did not work.
Applied Work Arround:
In-order to fix the issue post our Infrastructure outage, We did re-run the same version of Pega upgrade.sh / upgrade.bat Pega Script over the existing Schemas and re-deployed the Application worked actually. E.g. We were using the 8.6.3 Platform and Same version of Pega Upgrade Script has been invoked and post the Script Completion, Probing Issues were resolved and the Application was back online with No changes in the Probing Values adjusted.
Old Status:
INC-225663 & INC-230280 raised with Pega GCS to find out the Root Cause. Permanent Resolution will be posted back here once it's concluded.
RCA:
It's been identified that, In the Database end, extension between btree and gist type is missing.
It's mandatory in the postgreSQL database so created the required extension manually using CREATE EXTENSION btree_gist; query.
We have also created an Index using the below query over pr_engineclasses fixed the issue today after 8 months of Struggle.
So Far: In-order to fix the Probing issues, Amending the Startup probing parameters in the Helm Deployment file / Values file is of no use. We have tried multiple times. In-fact, Increased the Startup Probe values to 6Hours and tried but did not work.
Applied Work Arround:
In-order to fix the issue post our Infrastructure outage, We did re-run the same version of Pega upgrade.sh / upgrade.bat Pega Script over the existing Schemas and re-deployed the Application worked actually. E.g. We were using the 8.6.3 Platform and Same version of Pega Upgrade Script has been invoked and post the Script Completion, Probing Issues were resolved and the Application was back online with No changes in the Probing Values adjusted.
Old Status:
INC-225663 & INC-230280 raised with Pega GCS to find out the Root Cause. Permanent Resolution will be posted back here once it's concluded.
RCA:
It's been identified that, In the Database end, extension between btree and gist type is missing.
It's mandatory in the postgreSQL database so created the required extension manually using CREATE EXTENSION btree_gist; query.
We have also created an Index using the below query over pr_engineclasses fixed the issue today after 8 months of Struggle.
CREATE INDEX <INDEX_NAME> ON <Rules-Schema>.PR_ENGINECLASSES ( PZCLASS, PZJAR, PZCODESET, PZCODESETVERSION, PZPATCHDATE, PZPACKAGE );
Note: Before applying the Work Arroud We identified to fix the issue, Make sure you take the necessary back up of DB and respective Deployment and config files.
I'll post the Reference Knowledge Article From Pega once it's published in Pega Community.