Exception when committing to database other than PegaDATA
Hello everyone,
In our Pega 7.1.7 with Split Schema environment we have another Database Instance configured.
When trying to write instances into tables of that database, we receive a strange error on the UI and see in tracer, that there's an exception while performing the commit operation:
Attached is the tracer file and here's the Error information from line 445:
Hello everyone,
In our Pega 7.1.7 with Split Schema environment we have another Database Instance configured.
When trying to write instances into tables of that database, we receive a strange error on the UI and see in tracer, that there's an exception while performing the commit operation:
Attached is the tracer file and here's the Error information from line 445:
[MSG][Encountered problem when performing a Commit][STACK][java.lang.NullPointerException at com.pega.pegarules.data.internal.access.Saver.addBatchSaver.java:967 at com.pega.pegarules.data.internal.access.Saver.saveInstanceSaver.java:825 at com.pega.pegarules.data.internal.access.Committer.performOpsCommitter.java:236 at com.pega.pegarules.data.internal.access.DatabaseImpl.attemptToProcessUpdatesDatabaseImpl.java:2625 at com.pega.pegarules.data.internal.access.DatabaseImpl.processUpdatesDatabaseImpl.java:2288 at com.pega.pegarules.data.internal.access.DatabaseImpl.commitDatabaseImpl.java:2027 at com.pegarules.generated.activity.ra_action_pxcreaterecordinner_624b4eca67cbed98f8f956241ca4a677.step11_circum0ra_action_pxcreaterecordinner_624b4eca67cbed98f8f956241ca4a677.java:1176 at com.pegarules.generated.activity.ra_action_pxcreaterecordinner_624b4eca67cbed98f8f956241ca4a677.performra_action_pxcreaterecordinner_624b4eca67cbed98f8f956241ca4a677.java:245 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pegarules.generated.activity.ra_action_pxcreaterecord_34ff83abccf5877184c3c1b19ef9136c.step2_circum0ra_action_pxcreaterecord_34ff83abccf5877184c3c1b19ef9136c.java:290 at com.pegarules.generated.activity.ra_action_pxcreaterecord_34ff83abccf5877184c3c1b19ef9136c.performra_action_pxcreaterecord_34ff83abccf5877184c3c1b19ef9136c.java:86 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivityExecutable.java:10450 at com.pegarules.generated.activity.ra_action_pxupdaterecordinner_5c47167c53f1f90911a336e47d50c720.step6_circum0ra_action_pxupdaterecordinner_5c47167c53f1f90911a336e47d50c720.java:861 at com.pegarules.generated.activity.ra_action_pxupdaterecordinner_5c47167c53f1f90911a336e47d50c720.performra_action_pxupdaterecordinner_5c47167c53f1f90911a336e47d50c720.java:140 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pegarules.generated.activity.ra_action_pxupdaterecord_870867102c85288d291629e62bd934a2.step2_circum0ra_action_pxupdaterecord_870867102c85288d291629e62bd934a2.java:290 at com.pegarules.generated.activity.ra_action_pxupdaterecord_870867102c85288d291629e62bd934a2.performra_action_pxupdaterecord_870867102c85288d291629e62bd934a2.java:86 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivityExecutable.java:10450 at com.pegarules.generated.activity.ra_action_pzupdaterecord_b1bd105138c54f2657e053e38a9d312a.step8_circum0ra_action_pzupdaterecord_b1bd105138c54f2657e053e38a9d312a.java:743 at com.pegarules.generated.activity.ra_action_pzupdaterecord_b1bd105138c54f2657e053e38a9d312a.performra_action_pzupdaterecord_b1bd105138c54f2657e053e38a9d312a.java:200 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivityExecutable.java:10450 at com.pegarules.generated.activity.ra_action_pzpostupdaterecordwrapper_d6dee74aacbf996ce049887bae3737ee.step8_circum0ra_action_pzpostupdaterecordwrapper_d6dee74aacbf996ce049887bae3737ee.java:186 at com.pegarules.generated.activity.ra_action_pzpostupdaterecordwrapper_d6dee74aacbf996ce049887bae3737ee.performra_action_pzpostupdaterecordwrapper_d6dee74aacbf996ce049887bae3737ee.java:69 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pegarules.generated.activity.ra_action_pzdogridaction_84f3e7348d7bf2bad1ec28065f8afa87.step5_circum0ra_action_pzdogridaction_84f3e7348d7bf2bad1ec28065f8afa87.java:825 at com.pegarules.generated.activity.ra_action_pzdogridaction_84f3e7348d7bf2bad1ec28065f8afa87.performra_action_pzdogridaction_84f3e7348d7bf2bad1ec28065f8afa87.java:137 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pegarules.generated.activity.ra_action_reloadsection_8d636d6ea3d6b9b680f77d0cd1f195dd.step5_circum0ra_action_reloadsection_8d636d6ea3d6b9b680f77d0cd1f195dd.java:730 at com.pegarules.generated.activity.ra_action_reloadsection_8d636d6ea3d6b9b680f77d0cd1f195dd.performra_action_reloadsection_8d636d6ea3d6b9b680f77d0cd1f195dd.java:137 at com.pega.pegarules.session.internal.mgmt.Executable.doActivityExecutable.java:3375 at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAltThreadRunner.java:641 at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivitiesAltPRThreadImpl.java:435 at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.runActivitiesHttpAPI.java:3252 at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInnerEngineAPI.java:384 at sun.reflect.GeneratedMethodAccessor112.invokeUnknown Source at sun.reflect.DelegatingMethodAccessorImpl.invokeDelegatingMethodAccessorImpl.java:37 at java.lang.reflect.Method.invokeMethod.java:611 at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLockPRSessionProviderImpl.java:1188 at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLockedPRSessionProviderImpl.java:926 at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLockedPRSessionProviderImpl.java:811 at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestEngineAPI.java:330 at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.invokeHttpAPI.java:839 at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privactEngineImpl.java:315 at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngineEngineImpl.java:263 at com.pega.pegarules.session.internal.engineinterface.etier.ejb.EngineBean.invokeEngineEngineBean.java:225 at sun.reflect.GeneratedMethodAccessor95.invokeUnknown Source at sun.reflect.DelegatingMethodAccessorImpl.invokeDelegatingMethodAccessorImpl.java:37 at java.lang.reflect.Method.invokeMethod.java:611 at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPRBootstrap.java:338 at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowablePRBootstrap.java:379 at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowableAppServerBridgeToPega.java:216 at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingExceptionAppServerBridgeToPega.java:238 at com.pega.pegarules.internal.etier.ejb.EngineBeanBoot.invokeEngineEngineBeanBoot.java:168 at com.pega.pegarules.internal.etier.interfaces.EJSLocalStatelessEngineBMT_f2439d86.invokeEngineUnknown Source at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngineInnerJNDIEnvironment.java:278 at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngineJNDIEnvironment.java:223 at com.pega.pegarules.web.impl.WebStandardImpl.makeEtierRequestWebStandardImpl.java:485 at com.pega.pegarules.web.impl.WebStandardImpl.doPostWebStandardImpl.java:290 at sun.reflect.GeneratedMethodAccessor109.invokeUnknown Source at sun.reflect.DelegatingMethodAccessorImpl.invokeDelegatingMethodAccessorImpl.java:37 at java.lang.reflect.Method.invokeMethod.java:611 at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPRBootstrap.java:338 at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowablePRBootstrap.java:379 at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowableAppServerBridgeToPega.java:216 at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodAppServerBridgeToPega.java:265 at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doPostWebStandardBoot.java:118 at javax.servlet.http.HttpServlet.serviceHttpServlet.java:595 at javax.servlet.http.HttpServlet.serviceHttpServlet.java:668 at com.ibm.ws.webcontainer.servlet.ServletWrapper.serviceServletWrapper.java:1230 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequestServletWrapper.java:779 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequestServletWrapper.java:478 at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequestServletWrapperImpl.java:178 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFiltersWebAppFilterManager.java:1111 at com.ibm.ws.webcontainer.webapp.WebApp.handleRequestWebApp.java:3815 at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequestWebGroup.java:304 at com.ibm.ws.webcontainer.WebContainer.handleRequestWebContainer.java:981 at com.ibm.ws.webcontainer.WSWebContainer.handleRequestWSWebContainer.java:1662 at com.ibm.ws.webcontainer.channel.WCChannelLink.readyWCChannelLink.java:200 at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscriminationHttpInboundLink.java:456 at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequestHttpInboundLink.java:518 at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequestHttpInboundLink.java:309 at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.completeHttpICLReadCallback.java:84 at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.completeSSLReadServiceContext.java:1818 at com.ibm.ws.tcp.channel.impl.WorkQueueManager.requestCompleteWorkQueueManager.java:558 at com.ibm.ws.tcp.channel.impl.WorkQueueManager.attemptIOWorkQueueManager.java:608 at com.ibm.ws.tcp.channel.impl.WorkQueueManager.workerRunWorkQueueManager.java:985 at com.ibm.ws.tcp.channel.impl.WorkQueueManager$Worker.runWorkQueueManager.java:1074 at com.ibm.ws.util.ThreadPool$Worker.runThreadPool.java:1720 ]
What are possible reasons for this behaviour? I'm confident we are doing somthing wrong or have somthing configured wrong. What should I or my team check in order to be able to also write instances to the other table not just read.
Thank you for any hint!
Accepted Solution
Updated: 13 Aug 2015 12:02 EDT
Pegasystems Inc.
US
We have seen this type of issue before where there is actually a database exception being thrown which goes to the Saver.addBatch method's catch block but the catch block is failing with a Null Pointer Exception prior to actually logging the database error message. I expect that the app server is complaining that they are not using an XA driver. You might also look in the app server's log as it might tell you the actual exception that is being thrown. If you don't see an error there there is a diagnostic hot fix where we can get the actual database error that is being thrown.
Accepted Solution
Updated: 13 Aug 2015 12:02 EDT
Pegasystems Inc.
US
We have seen this type of issue before where there is actually a database exception being thrown which goes to the Saver.addBatch method's catch block but the catch block is failing with a Null Pointer Exception prior to actually logging the database error message. I expect that the app server is complaining that they are not using an XA driver. You might also look in the app server's log as it might tell you the actual exception that is being thrown. If you don't see an error there there is a diagnostic hot fix where we can get the actual database error that is being thrown.
Updated: 14 Aug 2015 3:24 EDT
Infosys
US
Hi,
I've requested this information from our platform management team. This is what I got:
The database is
Oracle Database 11g Enterprise Edition Release Proprietary information hidden.0 - 64bit Production With the Partitioning and Automatic Storage Management options
and they provided those screenshots from the Websphere configuration console:
Infosys
US
The app server log file seems to suggest you are on the right track, Celeste and Gangababu.
Here's the (relavant) content, the complete version is attached:
[8/14/15 9:41:09:916 CEST] 00000043 RegisteredRes E WTRN0062E: An illegal attempt to use multiple resources that have only one-phase capability has occurred within a global transaction.
[8/14/15 9:41:09:982 CEST] 00000043 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /var/opt/log/websphere/n1pega80a/ffdc/was5member01_9b501bf7_15.08.14_ Proprietary information hidden8063053519751749059.txt com.ibm.ws.Transaction.JTA.TransactionImpl.enlistResource 1858
[8/14/15 9:41:09:987 CEST] 00000043 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /var/opt/log/websphere/n1pega80a/ffdc/was5member01_9b501bf7_15.08.14_ Proprietary information hidden6549726544072375598.txt com.ibm.ejs.j2c.LocalTransactionWrapper.enlist 459
[8/14/15 9:41:09:987 CEST] 00000043 LocalTransact E J2CA0030E: Method enlist caught com.ibm.ws.Transaction.IllegalResourceIn2PCTransactionException: Illegal attempt to enlist multiple 1PC XAResources
at com.ibm.ws.tx.jta.RegisteredResources.enlistResourceRegisteredResources.java:871
at com.ibm.ws.tx.jta.TransactionImpl.enlistResourceTransactionImpl.java:1849
at com.ibm.tx.jta.impl.EmbeddableTranManagerSet.enlistOnePhaseEmbeddableTranManagerSet.java:195
The app server log file seems to suggest you are on the right track, Celeste and Gangababu.
Here's the (relavant) content, the complete version is attached:
[8/14/15 9:41:09:916 CEST] 00000043 RegisteredRes E WTRN0062E: An illegal attempt to use multiple resources that have only one-phase capability has occurred within a global transaction.
[8/14/15 9:41:09:982 CEST] 00000043 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /var/opt/log/websphere/n1pega80a/ffdc/was5member01_9b501bf7_15.08.14_ Proprietary information hidden8063053519751749059.txt com.ibm.ws.Transaction.JTA.TransactionImpl.enlistResource 1858
[8/14/15 9:41:09:987 CEST] 00000043 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /var/opt/log/websphere/n1pega80a/ffdc/was5member01_9b501bf7_15.08.14_ Proprietary information hidden6549726544072375598.txt com.ibm.ejs.j2c.LocalTransactionWrapper.enlist 459
[8/14/15 9:41:09:987 CEST] 00000043 LocalTransact E J2CA0030E: Method enlist caught com.ibm.ws.Transaction.IllegalResourceIn2PCTransactionException: Illegal attempt to enlist multiple 1PC XAResources
at com.ibm.ws.tx.jta.RegisteredResources.enlistResourceRegisteredResources.java:871
at com.ibm.ws.tx.jta.TransactionImpl.enlistResourceTransactionImpl.java:1849
at com.ibm.tx.jta.impl.EmbeddableTranManagerSet.enlistOnePhaseEmbeddableTranManagerSet.java:195
at com.ibm.ejs.j2c.LocalTransactionWrapper.enlistLocalTransactionWrapper.java:607
at com.ibm.ejs.j2c.ConnectionManager.lazyEnlistConnectionManager.java:2755
at com.ibm.ws.rsadapter.spi.WSRdbManagedConnectionImpl.lazyEnlistWSRdbManagedConnectionImpl.java:2596
at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.beginTransactionIfNecessaryWSJdbcConnection.java:732
at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.prepareStatementWSJdbcConnection.java:2786
at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.prepareStatementWSJdbcConnection.java:2739
at com.pega.pegarules.data.internal.access.DatabaseConnectionImpl.prepareStatementDatabaseConnectionImpl.java:518
[...]
at com.ibm.ws.util.ThreadPool$Worker.runThreadPool.java:1720
while trying to enlist resources from DataSource jdbc/PegaRULES with the Transaction Manager for the current transaction, and threw a Exception.
[8/14/15 9:41:10:008 CEST] 00000043 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /var/opt/log/websphere/n1pega80a/ffdc/was5member01_9b501bf7_15.08.14_ Proprietary information hidden6857051484333278667.txt com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.beginTransactionIfNecessary 324
I'll try to convince plattform management to give an XA Driver a try and will report back afterwards.
Thank you very much for the support up until now!
Infosys
US
We are currently not able to change the database driver.
I'll mark your suggestion as correct as it seems using a XA driver solves our problem.
Once we can give that solution a try, I'll reopen this discussion should we have ongoing issues.
Thank You very much for pointing us into the right direction.