Skip to main content

This content is closed to future replies and is no longer being maintained or updated.
Links may no longer function. If you have a similar request, please write a new post.

Question

 Ashutosh Singh (Ashutoshsingh)
BlueRose Technologies
Pega Lead System Architect
BlueRose Technologies
NL
Ashutoshsingh Member since 2012 8 posts
BlueRose Technologies
Posted: Mar 2, 2017
Last activity: Aug 10, 2017
Posted: 2 Mar 2017 10:26 EST
Last activity: 10 Aug 2017 6:00 EDT
Closed
Solved

Production server got down and logs filled with these entries

2017-01-03 00:02:31,316 [.PegaWorkManager : 1] [ STANDARD] [ ] [ FPM:01.02] ( internal.async.AgentQueue) ERROR - Agent Pega-RULES #2: Code-.SystemIndexer encountered an execution error. Restarting after delay.
java.lang.NullPointerException
at com.pega.pegarules.session.internal.mgmt.Executable.runIncrementalIndexer(Executable.java:11105)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.step2_circum0(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:234)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.perform(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:86)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3505)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.runActivity(QueueProcessor.java:721)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.execute(QueueProcessor.java:602)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:945)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1259)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1008)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:901)

Show More

2017-01-03 00:02:31,316 [.PegaWorkManager : 1] [ STANDARD] [ ] [ FPM:01.02] ( internal.async.AgentQueue) ERROR - Agent Pega-RULES #2: Code-.SystemIndexer encountered an execution error. Restarting after delay.
java.lang.NullPointerException
at com.pega.pegarules.session.internal.mgmt.Executable.runIncrementalIndexer(Executable.java:11105)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.step2_circum0(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:234)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.perform(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:86)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3505)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.runActivity(QueueProcessor.java:721)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.execute(QueueProcessor.java:602)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:945)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1259)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1008)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:901)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:670)
at com.pega.pegarules.session.internal.async.Agent.queueBatchActivityForAgent(Agent.java:1504)
at com.pega.pegarules.session.internal.async.Agent.queueBatchActivityForAgent(Agent.java:1293)
at com.pega.pegarules.session.internal.async.AgentQueue.run(AgentQueue.java:792)
at com.pega.pegarules.session.internal.async.Agent.runAgentTask(Agent.java:936)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.AsyncServant.processAsyncTask(AsyncServant.java:221)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.AsyncServant.invoke(AsyncServant.java:167)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:315)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:263)
at com.pega.pegarules.session.internal.engineinterface.etier.ejb.EngineBean.invokeEngine(EngineBean.java:225)
at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:370)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:411)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:223)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingException(AppServerBridgeToPega.java:245)
at com.pega.pegarules.internal.etier.ejb.EngineBeanBoot.invokeEngine(EngineBeanBoot.java:168)
at com.pega.pegarules.internal.etier.interfaces.EJSLocalStatelessEngineBMT_f2439d86.invokeEngine(Unknown Source)
at com.pega.pegarules.session.internal.engineinterface.etier.mdb.PRAsync.processRequest(PRAsync.java:155)
at com.pega.pegarules.session.internal.engineinterface.etier.asynchbeans.AsyncTask.run(AsyncTask.java:64)
at com.pega.pegarules.session.internal.engineinterface.etier.asynchbeans.RunnableWrapper.run(RunnableWrapper.java:71)
at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.java:270)
at java.security.AccessController.doPrivileged(AccessController.java:285)
at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:796)
at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:222)
at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.java:206)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)
2017-01-03 00:02:31,349 [.PegaWorkManager : 1] [ STANDARD] [ ] [ xx:01.02] ( internal.async.AgentQueue) ERROR - Problem queue Pega-RULES #2: Code-.SystemIndexer will restart in 240000 ms
2017-01-03 00:02:40,801 [PegaWorkManager : 45] [ STANDARD] [ ] [ xx:01.02] ( internal.mgmt.Executable) ERROR - Exception
java.lang.NullPointerException
at com.pega.pegarules.session.internal.mgmt.Executable.runIncrementalIndexer(Executable.java:11105)
at com.pegarules.generated.activity.ra_action_systemworkindexer_f75cf336649533591314b343e81e15eb.step2_circum0(ra_action_systemworkindexer_f75cf336649533591314b343e81e15eb.java:213)
at com.pegarules.generated.activity.ra_action_systemworkindexer_f75cf336649533591314b343e81e15eb.perform(ra_action_systemworkindexer_f75cf336649533591314b343e81e15eb.java:86)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3505)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.runActivity(QueueProcessor.java:721)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.execute(QueueProcessor.java:602)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:945)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1259)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1008)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:901)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:670)
at com.pega.pegarules.session.internal.async.Agent.queueBatchActivityForAgent(Agent.java:1504)
at com.pega.pegarules.session.internal.async.Agent.queueBatchActivityForAgent(Agent.java:1293)
at com.pega.pegarules.session.internal.async.AgentQueue.run(AgentQueue.java:792)
at com.pega.pegarules.session.internal.async.Agent.runAgentTask(Agent.java:936)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.AsyncServant.processAsyncTask(AsyncServant.java:221)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.AsyncServant.invoke(AsyncServant.java:167)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:315)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:263)
at com.pega.pegarules.session.internal.engineinterface.etier.ejb.EngineBean.invokeEngine(EngineBean.java:225)
at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:370)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:411)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:223)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingException(AppServerBridgeToPega.java:245)
at com.pega.pegarules.internal.etier.ejb.EngineBeanBoot.invokeEngine(EngineBeanBoot.java:168)
at com.pega.pegarules.internal.etier.interfaces.EJSLocalStatelessEngineBMT_f2439d86.invokeEngine(Unknown Source)
at com.pega.pegarules.session.internal.engineinterface.etier.mdb.PRAsync.processRequest(PRAsync.java:155)
at com.pega.pegarules.session.internal.engineinterface.etier.asynchbeans.AsyncTask.run(AsyncTask.java:64)
at com.pega.pegarules.session.internal.engineinterface.etier.asynchbeans.RunnableWrapper.run(RunnableWrapper.java:71)
at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.java:270)
at java.security.AccessController.doPrivileged(AccessController.java:285)
at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:796)
at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:222)
at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.java:206)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)

***Updated by Moderator: Marissa to add SR details***


Show Less
To see attachments, please log in.
System Administration
Support Case Created

Accepted Solution

Posted: 8 years ago
Posted: 6 May 2017 7:40 EDT
Domenico Giffone (DomenicoGiffone)
PEGA
Senior Principal Software Engineer, 1:1 Runtime
Pegasystems Inc.
IT

The exception reported here is not related to the experienced issue.

Root cause of the issue was an OutOfMemory error caused by a poor system configuration. In this case multiple Pega applications were running in the same Application Server instance.  As such memory requirements were not the same as in the single application deployment scenario described in the installation guide.

The issue has been resolved adjusting the memory assigned to the application according to the actual load observed on the system.

To see attachments, please log in.
Posted: 8 years ago
Posted: 2 Mar 2017 12:28 EST
Sudhish Othoyoth Puthiyapurayil (othos) Senior Technical Solutions Engineer
Pegasystems Inc.
IN

Hi Ashutosh,

I think threads and requestors during indexing was faulty and that might be the issue for these exception. What is  your PRPC version? For 6x we have hotfix for similar case and if this is happening in PRPC 7x then there are some of the most elusive issues to narrow down, and will almost always need an deep investigation in an SR to establish a true root cause.
   

 

To see attachments, please log in.
Posted: 8 years ago
Posted: 2 Mar 2017 13:05 EST
Arvind Malav (malaa) Technical Manager
Pegasystems Inc.
IN

Hi Ashutosh,

For Production down, it is advisable to call Support hotline.

W.r.t this post, I suggest to share complete log file when this happened.

To see attachments, please log in.
Posted: 8 years ago
Posted: 3 Mar 2017 2:25 EST
Lochana Durgada Vijayakumar (Lochan_DV)
PEGA
Senior Manager, Knowledge Management
Pegasystems Inc.
IN

Hi Ashutosh,

As it is determined to be a candidate for the Support Request, please let us know the SR number once you create it. We will tag that to this post for effective tracking.

Thank you!

To see attachments, please log in.

Accepted Solution

Posted: 8 years ago
Posted: 6 May 2017 7:40 EDT
Domenico Giffone (DomenicoGiffone)
PEGA
Senior Principal Software Engineer, 1:1 Runtime
Pegasystems Inc.
IT

The exception reported here is not related to the experienced issue.

Root cause of the issue was an OutOfMemory error caused by a poor system configuration. In this case multiple Pega applications were running in the same Application Server instance.  As such memory requirements were not the same as in the single application deployment scenario described in the installation guide.

The issue has been resolved adjusting the memory assigned to the application according to the actual load observed on the system.

To see attachments, please log in.
Posted: 8 years ago
Posted: 10 Aug 2017 6:00 EDT
narayana Yanamandra (NarayanaSastryY)
Bank Of America
Software Engineer
Bank Of America
IN

I am also having same kind of issue, i am getting too many null pointer exceptions like this

 

Show More

I am also having same kind of issue, i am getting too many null pointer exceptions like this

 

(root cause)
Type java.lang.NullPointerException
 
Message java.lang.NullPointerException
 
Stack 
at com.pega.pegarules.session.internal.mgmt.Executable.runIncrementalIndexer(Executable.java:11105)
at com.pegarules.generated.activity.ra_action_pzftsincrementalindexer_f5d34531e6ce3428b7cfcf71c389b182.step2_circum0(ra_action_pzftsincrementalindexer_f5d34531e6ce3428b7cfcf71c389b182.java:213)
at com.pegarules.generated.activity.ra_action_pzftsincrementalindexer_f5d34531e6ce3428b7cfcf71c389b182.perform(ra_action_pzftsincrementalindexer_f5d34531e6ce3428b7cfcf71c389b182.java:86)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3505)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.runActivity(QueueProcessor.java:721)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.execute(QueueProcessor.java:602)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:945)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1259)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1008)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:901)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:670)
at com.pega.jsr166backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:991)
at com.pega.jsr166backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:532)
at java.lang.Thread.run(Thread.java:745)
 
at com.pega.pegarules.session.internal.mgmt.Executable.runIncrementalIndexer(Executable.java:11105)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.step2_circum0(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:234)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.perform(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:86)
 
at com.pega.pegarules.session.internal.mgmt.Executable.runIncrementalIndexer(Executable.java:11105)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.step2_circum0(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:234)
at com.pegarules.generated.activity.ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.perform(ra_action_systemindexer_bcf109e229b3b868311b17d40c4cf617.java:86)

Show Less
To see attachments, please log in.

We'd prefer it if you saw us at our best.

Pega Collaboration Center has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice