Asynchronous File Listener
Hi,
My Requirement is to process a large csv file and update db tables, for that we are using Pega Filelistener, Performance wise it is very bad.
To improve performance i am running file listener asynchronously and by splitting queuing it to "System-Queue-ExecutionRequest-Service-Default" class and tried running ProcessImmediateQueueItems agent activity.
The agent fails to pick few queue items and in the logs i am getting the below logs.
2016-04-21 15:58:18,986 {ABSOLUTE} [PRPCWorkManager : 12] [ PegaRULES:07.10] (Queue_ExecutionRequest_.Action) ERROR - Failed to process Queue Item ID: SYSTEM-QUEUE-EXECUTIONREQUEST-SERVICE-DEFAULT 146125068244200078FEE8F7D1085916B39BA68D65C74929
com.pega.pegarules.pub.PRException: Caught exception while retrieving Service Request with id: SYSTEM-QUEUE-EXECUTIONREQUEST-SERVICE-DEFAULT 146125068244200078FEE8F7D1085916B39BA68D65C74929
From: (ABFB4B47E8C336CB93DC9C40080B225A4)
at com.pega.pegarules.session.internal.engineinterface.service.RequestProcessorImpl.processServiceRequest(RequestProcessorImpl.java:716)
at com.pegarules.generated.activity.ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.step1_circum0(ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.java:160)
at com.pegarules.generated.activity.ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.perform(ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.java:69)
Hi,
My Requirement is to process a large csv file and update db tables, for that we are using Pega Filelistener, Performance wise it is very bad.
To improve performance i am running file listener asynchronously and by splitting queuing it to "System-Queue-ExecutionRequest-Service-Default" class and tried running ProcessImmediateQueueItems agent activity.
The agent fails to pick few queue items and in the logs i am getting the below logs.
2016-04-21 15:58:18,986 {ABSOLUTE} [PRPCWorkManager : 12] [ PegaRULES:07.10] (Queue_ExecutionRequest_.Action) ERROR - Failed to process Queue Item ID: SYSTEM-QUEUE-EXECUTIONREQUEST-SERVICE-DEFAULT 146125068244200078FEE8F7D1085916B39BA68D65C74929
com.pega.pegarules.pub.PRException: Caught exception while retrieving Service Request with id: SYSTEM-QUEUE-EXECUTIONREQUEST-SERVICE-DEFAULT 146125068244200078FEE8F7D1085916B39BA68D65C74929
From: (ABFB4B47E8C336CB93DC9C40080B225A4)
at com.pega.pegarules.session.internal.engineinterface.service.RequestProcessorImpl.processServiceRequest(RequestProcessorImpl.java:716)
at com.pegarules.generated.activity.ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.step1_circum0(ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.java:160)
at com.pegarules.generated.activity.ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.perform(ra_action_processrequest_dd1c885224bcfc60d073c520b6142f7d.java:69)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:3375)
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:641)
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivities(ThreadRunner.java:568)
at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivities(PRThreadImpl.java:427)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.executeBatchTask(QueueProcessor.java:256)
at com.pega.pegarules.session.internal.async.agent.QueueProcessor.execute(QueueProcessor.java:319)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:885)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1177)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:926)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:825)
at com.pega.pegarules.session.internal.async.BatchRequestorTask.run(BatchRequestorTask.java:618)
at com.pega.pegarules.session.internal.async.Agent.runBatchTask(Agent.java:816)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.AsyncServant.processAsyncTask(AsyncServant.java:229)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.AsyncServant.invoke(AsyncServant.java:166)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:315)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:263)
at com.pega.pegarules.session.internal.engineinterface.etier.ejb.EngineBean.invokeEngine(EngineBean.java:225)
at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:338)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:379)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:216)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingException(AppServerBridgeToPega.java:238)
at com.pega.pegarules.internal.etier.ejb.EngineBeanBoot.invokeEngine(EngineBeanBoot.java:168)
at com.pega.pegarules.internal.etier.interfaces.EJSLocalStatelessEngineBMT_f2439d86.invokeEngine(Unknown Source)
at com.pega.pegarules.session.internal.engineinterface.etier.mdb.PRAsync.processRequest(PRAsync.java:155)
at com.pega.pegarules.session.internal.engineinterface.etier.asynchbeans.AsyncTask.run(AsyncTask.java:64)
at com.pega.pegarules.session.internal.engineinterface.etier.asynchbeans.RunnableWrapper.run(RunnableWrapper.java:71)
at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.java:268)
at java.security.AccessController.doPrivileged(AccessController.java:250)
at javax.security.auth.Subject.doAs(Subject.java:495)
at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:131)
at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:89)
at com.ibm.ws.asynchbeans.J2EEContext$DoAsProxy.run(J2EEContext.java:335)
at java.security.AccessController.doPrivileged(AccessController.java:277)
at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:777)
at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:222)
at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.java:159)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1646)
Caused by:
com.pega.pegarules.pub.context.PRQueueException: Unable to dequeue specific item: SYSTEM-QUEUE-EXECUTIONREQUEST-SERVICE-DEFAULT 146125068244200078FEE8F7D1085916B39BA68D65C74929
at com.pega.pegarules.session.internal.async.agent.QueueManagerImpl.dequeue(QueueManagerImpl.java:865)
at com.pega.pegarules.session.internal.async.agent.QueueManagerImpl.dequeue(QueueManagerImpl.java:668)
at com.pega.pegarules.session.internal.async.agent.QueueManagerImpl.dequeue(QueueManagerImpl.java:663)
at com.pega.pegarules.session.internal.async.agent.QueueManagerImpl.dequeue(QueueManagerImpl.java:656)
at com.pega.pegarules.session.internal.async.agent.QueueManagerImpl.dequeue(QueueManagerImpl.java:649)
at com.pega.pegarules.session.internal.engineinterface.service.RequestProcessorImpl.processServiceRequest(RequestProcessorImpl.java:704)
... 42 more
Caused by:
com.pega.pegarules.pub.database.AuthorizationException: You are not authorized to open instance SYSTEM-QUEUE-EXECUTIONREQUEST-SERVICE-DEFAULT 146125068244200078FEE8F7D1085916B39BA68D65C74929
From: (ABFB4B47E8C336CB93DC9C40080B225A4)
at com.pega.pegarules.pub.database.AuthorizationException.createCannotOpenException(AuthorizationException.java:206)
at com.pega.pegarules.session.internal.async.agent.QueueManagerImpl.dequeue(QueueManagerImpl.java:803)
... 47 more
Tetrasoft India Private Limited
IN
Hi Harish,
In case you are facing performance issue in above scenario, you may give a look to following PDN article (if suitable) :
PEG
IN
Is issue consistent that whenever you upload a large csv to process? If yes, please turn on the debug loggers on file listener class. It may give some clue.
com.pega.pegarules.integration.engine.internal.services.file.FileListener
Perficient
US
The key portion of the stack trace is this error message.
com.pega.pegarules.pub.database.AuthorizationException: You are not authorized to open instance SYSTEM-QUEUE-EXECUTIONREQUEST-SERVICE-DEFAULT
How is your agent configured? The requestor for the agent does not have the necessary permissions to process the queue item.
Unitedhealth Group, Inc.
US
Searching for anything about "com.pega.pegarules.pub.database.AuthorizationException" this is all I found;
in my instance I'm seeing this twice, and the system it's on is a fairly straight-forward, unmodified, PRPC 7.1.9.
I'm seeing this with this stack trace:
ERROR - Error in Obj-Open
com.pega.pegarules.pub.database.AuthorizationException: You are not authorized to open instance SYSTEM-STATUS-NODES GAA_DEV!APSRD5339.UHC.COM!A5E80EC5AD4C31B37FAF3BC8D3662D2B
From: (B82E6D807808DEF692B925DC9771B559D)
at com.pega.pegarules.pub.database.AuthorizationException.createCannotOpenException(AuthorizationException.java:236)
at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:4121)
at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:3800)
at com.pega.pegarules.pub.runtime.AbstractActivity.objOpen(AbstractActivity.java:80)
at com.pegarules.generated.activity.ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.step2_circum0(ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.java:281)
at com.pegarules.generated.activity.ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.perform(ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.java:89)
Searching for anything about "com.pega.pegarules.pub.database.AuthorizationException" this is all I found;
in my instance I'm seeing this twice, and the system it's on is a fairly straight-forward, unmodified, PRPC 7.1.9.
I'm seeing this with this stack trace:
ERROR - Error in Obj-Open
com.pega.pegarules.pub.database.AuthorizationException: You are not authorized to open instance SYSTEM-STATUS-NODES GAA_DEV!APSRD5339.UHC.COM!A5E80EC5AD4C31B37FAF3BC8D3662D2B
From: (B82E6D807808DEF692B925DC9771B559D)
at com.pega.pegarules.pub.database.AuthorizationException.createCannotOpenException(AuthorizationException.java:236)
at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:4121)
at com.pega.pegarules.data.internal.access.DatabaseImpl.open(DatabaseImpl.java:3800)
at com.pega.pegarules.pub.runtime.AbstractActivity.objOpen(AbstractActivity.java:80)
at com.pegarules.generated.activity.ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.step2_circum0(ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.java:281)
at com.pegarules.generated.activity.ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.perform(ra_action_pyupdateactiveusercount_339df135247b5679b05a086dcf422e30.java:89)
and more.
But where is this coming from? I haven't "configured" anything on this system other than OOTB agents and listeners (in other words, this is not related to a file listener I created, unlike the OP)
Pegasystems Inc.
US
It comes from this agent shown in the screenshot - the activity pyupdateactiveusercount is reading data from pr_status_nodes table - I suspect you may have some stale nodes in the table, can you inspect the records of that table?
