7.3 consistently crashing (IJ000453: Unable to get managed connection for java:/eis/PRAdapterConnectionFactory)
We upgraded our development servers from 7.2.1 to 7.3.0 and they have been consistently crashing and we can't keep them up for more than a day or so. After some period of time, we just can't make connections to the database anymore.
We get these repeatedly in the logs with no stack trace:
2017-11-06 01:02:28,358 [-threads-1435421382)] [ STANDARD] [ PegaRULES:07.10] ( pegarules.resadap.RAClient) ERROR - Caught exception while creating Connection, javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/eis/PRAdapterConnectionFactory
2017-11-06 04:27:38,663 [http- Proprietary information hidden:8080-16] [ STANDARD] [ Gattaca:01.01.01] ( pegarules.resadap.RAClient) ERROR XX.XX.XX.XX|XX.XX.XX.XX lclemons007 - Caught exception while creating Connection, javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/eis/PRAdapterConnectionFactory
Attempted fixes:
We upgraded our development servers from 7.2.1 to 7.3.0 and they have been consistently crashing and we can't keep them up for more than a day or so. After some period of time, we just can't make connections to the database anymore.
We get these repeatedly in the logs with no stack trace:
2017-11-06 01:02:28,358 [-threads-1435421382)] [ STANDARD] [ PegaRULES:07.10] ( pegarules.resadap.RAClient) ERROR - Caught exception while creating Connection, javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/eis/PRAdapterConnectionFactory
2017-11-06 04:27:38,663 [http- Proprietary information hidden:8080-16] [ STANDARD] [ Gattaca:01.01.01] ( pegarules.resadap.RAClient) ERROR XX.XX.XX.XX|XX.XX.XX.XX lclemons007 - Caught exception while creating Connection, javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/eis/PRAdapterConnectionFactory
Attempted fixes:
https://community.pega.com/support/support-articles/unable-get-managed-connection-javaeispradaptercf
In jboss-pradapter-ds.xml, we've added this line <max-pool-size>100</max-pool-size>
https://community.pega.com/support/support-articles/application-becomes-unresponsive-during-load-test
In our data source configuration, we've added both of these for PegaRulesAdmin
<min-pool-size>150</min-pool-size>
<max-pool-size>300</max-pool-size>
***Edited by Moderator Marissa to remove PI***
Accepted Solution
Pegasystems Inc.
AU
Hi Derek,
Connection pool you are pointing to in standalone-full.xml is for Database and the one you are changing in ironjacamar.xml is for the Async Requests. This again depends on the use case of how many DB Connections you need and How many Async requests running.
In general, i believe the numbers which you mentioned makes sense.
Note: This issue can be completely eliminated if the PEGA is deployed as WAR(if you are not seperately using any EJB components) as PRPC manages the threads using the standard Java threads or you can also implement using ActiveMQ instead of Hornet Q (Refer to https://docs-previous.pega.com/preventing-stopped-or-delayed-agents-jboss-ear-deployments-activemq-resource-adapter).
Thanks,
Karthik
Pegasystems Inc.
AU
Hi Derek,
You can also try changing the <consumer-window-size>10240</consumer-window-size> to <consumer-window-size> 0</consumer-window-size> for the hornetq-ra connection factory in stand-alone-full.xml.
The changes you have applied may have caused the overload. Change the resource adapter connection factory by following the PDN article: https://community.pega.com/support/support-articles/application-becomes-unresponsive-during-load-test . Set the min pool to 100 and max pool to 200.
If still not resolved, check with you JBoss Admin team who can suggest settings to be fin tuned as per your requirement.
Accenture Federal Services
US
Hello eppik,
Hello eppik,
I am working with my JBoss administrator and following your guidance. Can I ask what the relationship between standalone-full.xml and ironjacamar.xml is? ie. Should we set the min/maxes to be the same or ?
This is what I'll have configured in standalone-full.xml:
<datasource jndi-name="java:/jdbc/AdminPegaRULES" pool-name="AdminPegaRULES" enabled="true">
<connection-url>xxxxxxx</connection-url>
<driver>oracle</driver>
<pool>
<min-pool-size>100</min-pool-size>
<max-pool-size>200</max-pool-size>
</pool>
<validation>
<valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker"/>
<stale-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleStaleConnectionChecker"/>
<exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleExceptionSorter"/>
</validation>
<timeout>
<blocking-timeout-millis>30000</blocking-timeout-millis>
<idle-timeout-minutes>5</idle-timeout-minutes>
</timeout>
.....
And this in ironjacamar.xml
<?xml version="1.0" encoding="UTF-8"?>
<ironjacamar>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definition class-name="com.pega.pegarules.ra.spi.PRManagedConnectionFactoryImpl" enabled="true" jndi-name="eis/PRAdapterCF" pool-name="Connection" use-ccm="false" use-java-context="true">
<config-property name="asyncConnectionFactory">java:comp/env/jms/tcf/Async</config-property>
<config-property name="asyncQueue">java:comp/env/jms/topic/AsyncTrigger</config-property>
<pool>
<min-pool-size>100</min-pool-size>
<max-pool-size>200</max-pool-size>
<prefill>true</prefill>
<use-strict-min>true</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
</pool>
<security>
<application/>
</security>
<timeout>
<blocking-timeout-millis>30000</blocking-timeout-millis>
<idle-timeout-minutes>5</idle-timeout-minutes>
</timeout>
<validation>
<background-validation>true</background-validation>
<use-fast-fail>true</use-fast-fail>
</validation>
</connection-definition>
</connection-definitions>
</ironjacamar>
Accepted Solution
Pegasystems Inc.
AU
Hi Derek,
Connection pool you are pointing to in standalone-full.xml is for Database and the one you are changing in ironjacamar.xml is for the Async Requests. This again depends on the use case of how many DB Connections you need and How many Async requests running.
In general, i believe the numbers which you mentioned makes sense.
Note: This issue can be completely eliminated if the PEGA is deployed as WAR(if you are not seperately using any EJB components) as PRPC manages the threads using the standard Java threads or you can also implement using ActiveMQ instead of Hornet Q (Refer to https://docs-previous.pega.com/preventing-stopped-or-delayed-agents-jboss-ear-deployments-activemq-resource-adapter).
Thanks,
Karthik