Hi All,

We are frequently facing OOM exceptions in our system.

PFB the exception faced by us

2018-06-20 14:34:27,518 [ TP-Processor65] [ STANDARD] [ PRDP:01.01.02] ( internal.services.ServiceAPI) ERROR lntrs394|XX.XX.XX.XX|HTTP|RBCCMPRDPWork|NewWork|AddFlow|A8BB0EA62CF911114B9158F693E29957A - HTTP service [RBCCMPRDPWork][NewWork][AddFlow] failed:PRRuntimeError:com.pega.pegarules.pub.PRRuntimeError: PRRuntimeError
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:686)
at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivitiesAlt(PRThreadImpl.java:582)
at com.pega.pegarules.integration.engine.internal.services.ServiceAPI.runActivities(ServiceAPI.java:1533)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInner(EngineAPI.java:399)
at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1000)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:730)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequest(EngineAPI.java:333)
at com.pega.pegarules.integration.engine.internal.services.http.HTTPService.invoke(HTTPService.java:315)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:315)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:266)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:243)
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngineInner(JNDIEnvironment.java:280)
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngine(JNDIEnvironment.java:225)
at com.pega.pegarules.web.impl.WebStandardImpl.makeEtierRequest(WebStandardImpl.java:343)
at com.pega.pegarules.web.impl.WebStandardImpl.doPost(WebStandardImpl.java:236)
at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:348)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:389)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:438)
at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doPost(WebStandardBoot.java:102)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.OutOfMemoryError: Java heap space

PFB the configuration used by our system

PRPC version : 6.1 SP2

System configuration:

Red hat linux
Tomcat version 6.x
Server - 8 GB memory/ 64 bit system
Java version 1.6,JVM - 32 bit

We have the following queries

1) We are trying to increase heap space to mitigate the issue, PFB the current heap space configuration.

export JAVA_OPTS="-Xms1280m –Xmx1280m -XX:PermSize=64m -XX:MaxPermSize=256m -Dcom.sun.management.jmxremote=lntrs394.uklb.fg.rbc.com
-Dcom.sun.management.jmxremote.port=9004 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.awt.headless=true -Ddk.itp.spnego.config=$PEGA_HOME/spnego/spnego.properties"

To mitigate the heap space error, PFB the proposed configuration(increased in max heap space).

export JAVA_OPTS="-Xms1280m –Xmx2048m -XX:PermSize=64m -XX:MaxPermSize=256m -Dcom.sun.management.jmxremote=lntrs394.uklb.fg.rbc.com -Dcom.sun.management.jmxremote.port=9004 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.awt.headless=true -Ddk.itp.spnego.config=$PEGA_HOME/spnego/spnego.properties"

Will this configuration be optimal for the server

2) Is 64-bit JVM compatible with Pega 6.1 SP2( with 32 bit JVM our hands are tied increasing heap/memory space , so we want to go to 64 bit JVM which would be much more appropriate to increase memory and heap space).

3) We are currently only having a two node system, will introducing a new node mitigate this issue as we think the no of requests coming into the system has increased.

Thanks in advance.