Skip to main content

This content is closed to future replies and is no longer being maintained or updated.
Links may no longer function. If you have a similar request, please write a new post.

Question

 John Paul Raja Christu Raja (JohnPaulRaja,C)
Maantic Inc
Consultant
Maantic Inc
US
JohnPaulRaja,C Member since 2011 307 posts
Maantic Inc
Posted: May 23, 2023
Last activity: Feb 9, 2024
Posted: 23 May 2023 19:07 EDT
Last activity: 9 Feb 2024 7:08 EST
Closed
Solved

pyBatchIndexProcessor fails to run

 
pyBatchIndexProcessor fails to run 

pyBatchIndexProcessor dataflow shows error 

Slow component [OutputStage] with id [Destination] on data flow [pyBatchIndexProcessor] belonging to class [System-Message-QueueProcessor] detected. 
Please check the logs of node [9e85a7e0b971aad3f64d4f60e19d8005] or the corresponding PEGA0131 alert for full details of the slow dataflow processing thread stacktrace. Stacktrace ID [f9ede8a5-a753-4eb4-bb88-a560e5671f90]

Anybody had this issue?

To see attachments, please log in.
Pega Platform 8.8.2
Pega Platform
Admin Studio

Accepted Solution

Posted: 2 years ago
Updated: 2 years ago
Posted: 26 Jan 2024 8:25 EST
Updated: 9 Feb 2024 7:08 EST
Marije Schillern (MarijeSchillern)
MOD
Senior Knowledge Management Specialist
Pegasystems Inc.
GB

@JohnPaulRaja,C  @AnshumanM1959 

Please can you go through the available documentation:

pyFTSIncrementalIndexer not running or turning off automatically

Stopping or pausing search reindexing

Managing queue processors

PEGA0131 alert: Detected slow Data Flow component

sy

From the FAQ:

Why is my new queue processor not working?

If your queue processor does not work, consider the following:

Show More

@JohnPaulRaja,C  @AnshumanM1959 

Please can you go through the available documentation:

pyFTSIncrementalIndexer not running or turning off automatically

Stopping or pausing search reindexing

Managing queue processors

PEGA0131 alert: Detected slow Data Flow component

sy

From the FAQ:

Why is my new queue processor not working?

If your queue processor does not work, consider the following:

  • Queue processor rules rely on the DSM service. In Pega Cloud services and client-managed cloud environments, a stream node is automatically configured and ready to use. For on-premises environments, ensure that you define at least one node in a cluster to be a Stream node using the setting: -DNodeType=Stream. For more information, see Stream node configurations for a queue processor rule.
  • One problem may be that Pega Platform is not be able to properly resolve your queue processor rule. For more information on how queue processor rules work, see System Runtime Context and General system settings.
  • If further troubleshooting is necessary, see Troubleshooting background processing.

There is also a good Pega Academy course: Background process management

Show Less
To see attachments, please log in.
Posted: 2 years ago
Posted: 12 Jan 2024 12:50 EST
Anshuman Mishra (AnshumanM1959)
Citigroup Inc

Citigroup Inc
CA

a@JohnPaulRaja,C I am al.so facing the same issue in Pega 8.8.4 . is this issue resolved.

 

Show More

a@JohnPaulRaja,C I am al.so facing the same issue in Pega 8.8.4 . is this issue resolved.

 

com.pega.platform.executor.queueprocessor.QueueProcessorRuntimeException: Unexpected exception while executing QP: 
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.RequeueErrorHandler.handle(RequeueErrorHandler.java:44)
	at com.pega.dsm.dnode.api.dataflow.service.ResilienceConfig$CustomErrorHandlerWrapper.handle(ResilienceConfig.java:439)
	at com.pega.dsm.dnode.impl.dataflow.resilience.AtLeastOnceResilienceStrategy.handleErrors(AtLeastOnceResilienceStrategy.java:335)
	at com.pega.dsm.dnode.impl.dataflow.resilience.AtLeastOnceResilienceStrategy.onError(AtLeastOnceResilienceStrategy.java:202)
	at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$4.process(DataFlowExecutor.java:371)
	at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor.runEventLoop(DataFlowExecutor.java:218)
	at com.pega.dsm.dnode.api.dataflow.DataFlow$2.emit(DataFlow.java:339)
	at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:59)
	at com.pega.dsm.dnode.impl.dataflow.task.strategy.ExecutionStrategy.executeDataFlow(ExecutionStrategy.java:122)
	at com.pega.dsm.dnode.impl.dataflow.task.strategy.MultiplePartitionExecution.executePartitions(MultiplePartitionExecution.java:87)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.executeDataFlowWithPartitions(ExecutePartitionsRunnable.java:86)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.access$900(ExecutePartitionsRunnable.java:34)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable$1.run(ExecutePartitionsRunnable.java:67)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable$1.run(ExecutePartitionsRunnable.java:62)
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:305)
	at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.call(ExecutePartitionsRunnable.java:62)
	at com.pega.dsm.dnode.impl.dataflow.service.run.ExecutePartitionsRunnable.call(ExecutePartitionsRunnable.java:34)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:69)
	at com.pega.dsm.dnode.util.PrpcRunnable$1.run(PrpcRunnable.java:66)
	at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77)
	at com.pega.dsm.dnode.impl.prpc.PrpcThreadFactory$PrpcThread.run(PrpcThreadFactory.java:164)
Caused by: com.pega.dsm.dnode.api.dataflow.StageException: Exception in stage: OutputStage
	at com.pega.dsm.dnode.api.dataflow.StageException.create(StageException.java:54)
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:504)
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:426)
	at com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$2.process(DataFlowExecutor.java:329)
	... 23 more
Caused by: com.pega.dsm.dnode.api.ExceptionWithInputRecord: com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorTransactionFailedException: PRRuntimeException
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:503)
	... 25 more
Caused by: com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorTransactionFailedException: PRRuntimeException
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorOnlyOnceExecutionStrategy.saveOffsetAndCommit(QueueProcessorOnlyOnceExecutionStrategy.java:297)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorOnlyOnceExecutionStrategy.after(QueueProcessorOnlyOnceExecutionStrategy.java:225)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorUsageReportingExecutionStrategy.after(QueueProcessorUsageReportingExecutionStrategy.java:88)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorUsageReportingExecutionStrategy.lambda$execute$0(QueueProcessorUsageReportingExecutionStrategy.java:66)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124)
	at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1005)
	at com.pega.pegarules.session.internal.PRSessionProviderImplForModules.doWithRequestorLocked(PRSessionProviderImplForModules.java:83)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorUsageReportingExecutionStrategy.execute(QueueProcessorUsageReportingExecutionStrategy.java:52)
	at com.pega.platform.executor.internal.LogContextDecorator.runInDecoratedScope(LogContextDecorator.java:38)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorDataFlowFactory$1$1.onNext(QueueProcessorDataFlowFactory.java:96)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorDataFlowFactory$1$1.onNext(QueueProcessorDataFlowFactory.java:79)
	at com.pega.dsm.dnode.api.dataflow.DataFlowStage$StageInputSubscriber.onNext(DataFlowStage.java:501)
	... 25 more
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
	at com.pega.pegarules.data.internal.access.DatabaseImpl.commit(DatabaseImpl.java:2210)
	at com.pega.pegarules.data.internal.access.DatabaseImpl.commit(DatabaseImpl.java:2137)
	at com.pega.dsm.dnode.impl.dataflow.asyncexecutor.QueueProcessorOnlyOnceExecutionStrategy.saveOffsetAndCommit(QueueProcessorOnlyOnceExecutionStrategy.java:291)
	... 37 more
Caused by: java.lang.NullPointerException
	at com.pega.pegarules.priv.LogHelper.isDebugEnabled(LogHelper.java:1097)
	at com.pega.pegarules.generation.internal.cache.AssemblyCacheWrapper.noteChange(AssemblyCacheWrapper.java:817)
	at com.pega.pegarules.generation.internal.PRGenProviderImpl.noteChangeABAOnly(PRGenProviderImpl.java:523)
	at com.pega.pegarules.generation.internal.assembly.FUAManagerImpl.noteChange(FUAManagerImpl.java:3493)
	at com.pega.pegarules.generation.internal.PRGenProviderImpl.noteChange(PRGenProviderImpl.java:494)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.noteChange(UpdateProcessor.java:1249)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.notifyOtherSystemsAfterUpdates(UpdateProcessor.java:1010)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.notifyEverybodyOfChanges(UpdateProcessor.java:528)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.after(UpdateProcessor.java:340)
	at com.pega.pegarules.data.internal.store.DataStoreManager.doInTransaction(DataStoreManager.java:296)
	at com.pega.pegarules.data.internal.access.UpdateProcessor.processUpdates(UpdateProcessor.java:233)
	at com.pega.pegarules.data.internal.access.DatabaseImpl.commit(DatabaseImpl.java:2175)
	... 39 more
Show Less
To see attachments, please log in.

Accepted Solution

Posted: 2 years ago
Updated: 2 years ago
Posted: 26 Jan 2024 8:25 EST
Updated: 9 Feb 2024 7:08 EST
Marije Schillern (MarijeSchillern)
MOD
Senior Knowledge Management Specialist
Pegasystems Inc.
GB

@JohnPaulRaja,C  @AnshumanM1959 

Please can you go through the available documentation:

pyFTSIncrementalIndexer not running or turning off automatically

Stopping or pausing search reindexing

Managing queue processors

PEGA0131 alert: Detected slow Data Flow component

sy

From the FAQ:

Why is my new queue processor not working?

If your queue processor does not work, consider the following:

Show More

@JohnPaulRaja,C  @AnshumanM1959 

Please can you go through the available documentation:

pyFTSIncrementalIndexer not running or turning off automatically

Stopping or pausing search reindexing

Managing queue processors

PEGA0131 alert: Detected slow Data Flow component

sy

From the FAQ:

Why is my new queue processor not working?

If your queue processor does not work, consider the following:

  • Queue processor rules rely on the DSM service. In Pega Cloud services and client-managed cloud environments, a stream node is automatically configured and ready to use. For on-premises environments, ensure that you define at least one node in a cluster to be a Stream node using the setting: -DNodeType=Stream. For more information, see Stream node configurations for a queue processor rule.
  • One problem may be that Pega Platform is not be able to properly resolve your queue processor rule. For more information on how queue processor rules work, see System Runtime Context and General system settings.
  • If further troubleshooting is necessary, see Troubleshooting background processing.

There is also a good Pega Academy course: Background process management

Show Less
To see attachments, please log in.

We'd prefer it if you saw us at our best.

Pega Collaboration Center has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice