Description

After performing an out-of-place upgrade with a single migration, PRPC fails to initialize. The error seen in catalina.out suggests of a missing table but does not specify what is missing. Nothing significant appears in PegaRULES.log at this time apart from an abrupt Shutdown message during initialization.

The Pega instance is being upgraded from 8.3.1 to 8.5.1 hosted on Tomcat 8.5 and Oracle 12c (remains unchanged).  All steps (listed below) were successful with no apparent problem observed.  App initialization fails post successful rule base upgrade.

Steps

Follow standard steps performing an out-of-place upgrade with a single migration.

https://community.pega.com/knowledgebase/documents/pega-platform-85-upgrade-guide-apache-tomcat-and-oracle

1. Disabling rule creation
2. Creating a new rules schema
3. Migrating to the new rules schema
4. Upgrading the rules schema from the command line
5. Granting access to the new rules schema tables
6. Upgrading the data schema
7. Improving performance by removing old rulesets
8. Defining default schemas after out-of-place upgrades
9. Redeploying the Pega Platform file

-- App fails to initialize at this point --

Error

Snippet from catalina.out:

03-Nov-2020 14:08:59.250 SEVERE [localhost-startStop-1] com.pega.pegarules.internal.bootstrap.PRBootstrap. Problem during method invocation (contextInitialized)
java.lang.ExceptionInInitializerError
at com.pega.pegarules.data.internal.PRDataProviderImpl.isMultiTenantEnvironment(PRDataProviderImpl.java:1421)
at com.pega.pegarules.exec.internal.basic.SystemSettingsImpl$SystemSettingsInner.<clinit>(SystemSettingsImpl.java:204)
at com.pega.pegarules.exec.internal.basic.SystemSettingsImpl.getInstance(SystemSettingsImpl.java:197)
at com.pega.pegarules.exec.internal.PRExecProviderImpl.getSystemSettings(PRExecProviderImpl.java:523)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageBase.<clinit>(ClipboardPageBase.java:183)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageFactoryImpl$ClipboardPageImplFactory.newProduct(ClipboardPageFactoryImpl.java:110)
at com.pega.pegarules.priv.factory.AbstractObjectFactory.acquireNewProduct(AbstractObjectFactory.java:351)
at com.pega.pegarules.priv.factory.AbstractObjectFactory.acquireObject(AbstractObjectFactory.java:427)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageFactoryImpl$ClipboardPageImplFactory.acquire(ClipboardPageFactoryImpl.java:97)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageFactoryImpl.createClipboardPage(ClipboardPageFactoryImpl.java:309)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageFactoryImpl.acquire(ClipboardPageFactoryImpl.java:154)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageFactoryImpl$StaticHelper.acquire(ClipboardPageFactoryImpl.java:334)
at com.pega.pegarules.data.external.clipboard.ClipboardPageFactory.acquire(ClipboardPageFactory.java:97)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineStartup.initClipboardPageFactory(EngineStartup.java:861)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineStartup.initEngine(EngineStartup.java:386)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._initEngine_privact(EngineImpl.java:180)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.doStartup(EngineImpl.java:152)
at com.pega.pegarules.web.servlet.WebAppLifeCycleListener._contextInitialized_privact(WebAppLifeCycleListener.java:210)
at com.pega.pegarules.web.servlet.AbstractLifeCycleListener._contextInitialized_privact(AbstractLifeCycleListener.java:145)
at com.pega.pegarules.web.servlet.AbstractLifeCycleListener.contextInitialized(AbstractLifeCycleListener.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:388)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:430)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:225)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethod(AppServerBridgeToPega.java:274)
at com.pega.pegarules.internal.web.servlet.WebAppLifeCycleListenerBoot.contextInitialized(WebAppLifeCycleListenerBoot.java:92)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:988)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1860)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.pega.pegarules.pub.PRRuntimeException: PRRuntimeException
at com.pega.pegarules.data.internal.access.PerformaneHelper.initiazeAndGet(PerformaneHelper.java:58)
at com.pega.pegarules.data.internal.access.PerformaneHelper.get(PerformaneHelper.java:47)
at com.pega.pegarules.data.internal.access.PerformanceCommand.generatePostProcessingQuery(PerformanceCommand.java:63)
at com.pega.pegarules.data.internal.access.PerformanceCommand.getPostProcessingQuery(PerformanceCommand.java:59)
at com.pega.pegarules.data.internal.access.PerformanceProcessor.postprocessing(PerformanceProcessor.java:75)
at com.pega.pegarules.data.internal.access.oracle.SQLGeneratorOracle.executeOptimizationForOracle(SQLGeneratorOracle.java:1453)
at com.pega.pegarules.data.internal.access.oracle.SQLGeneratorOracle.getColumnListFromQuery(SQLGeneratorOracle.java:1438)
at com.pega.pegarules.data.internal.access.oracle.SQLGeneratorOracle.getColumnList(SQLGeneratorOracle.java:1385)
at com.pega.pegarules.data.internal.store.TableInformationMapImpl.lookupColumnInfo(TableInformationMapImpl.java:661)
at com.pega.pegarules.data.internal.store.TableInformationMapImpl.lookupRdbTableInfo(TableInformationMapImpl.java:491)
at com.pega.pegarules.data.internal.store.TableInformationMapImpl.lookupTableInfo(TableInformationMapImpl.java:435)
at com.pega.pegarules.data.internal.store.TableInformationMapImpl.getDataStoreTableInfo(TableInformationMapImpl.java:368)
at com.pega.pegarules.data.internal.PRDataProviderImpl$MultiTenantEnvironmentResource.isMultiTenantEnvironment(PRDataProviderImpl.java:1449)
at com.pega.pegarules.data.internal.PRDataProviderImpl$MultiTenantEnvironmentResource.<clinit>(PRDataProviderImpl.java:1425)
... 42 more
Caused by: com.pega.pegarules.pub.database.DatabaseException: Database-General ORA-00942: table or view does not exist

From: (unknown)
at com.pega.pegarules.data.internal.access.PerformaneHelper.initialize(PerformaneHelper.java:81)
at com.pega.pegarules.data.internal.access.PerformaneHelper.initiazeAndGet(PerformaneHelper.java:55)
... 55 more

Root cause

In Pega 8.5.1 an issue with Oracle causes significant performance problem.

Resolution (Workaround)

To overcome it, Pega user requires at least select on v$parameter to get the Pega 851 engine started.

Either, Grant Pega Base user running the environment following privilege (role) to fix this issue:

GRANT SELECT_CATALOG_ROLE..;

Alternatively, you can limit granting following privileges to the Base user instead of granting above role.

GRANT SELECT ON SYS.V_$PARAMETER ..;
GRANT ALTER SESSION ..;

Pegasystems have been notified so the deployment guides will (soon) be updated to reflect the need for this privilege to workaround the existing issue.