Skip to main content

This content is closed to future replies and is no longer being maintained or updated.
Links may no longer function. If you have a similar request, please write a new post.

Question

 kasim ergun (kasime17008587)
sirar by stc

sirar by stc
SA
kasime17008587 Member since 2023 2 posts
sirar by stc
Posted: Dec 28, 2023
Last activity: Jan 30, 2024
Posted: 28 Dec 2023 9:41 EST
Last activity: 30 Jan 2024 10:51 EST
Closed
Solved

Background node does not start on On-premise Pega '23

Hi Everyone

After installed '23 successfully, we failed to start Pega as Stream node.I successfully addressed a Stream node issue by adding <env name="services/stream/provider" value="Internal"/> to prconfig.xml . Stream node is running.

However still are facing another issue on Search and BackgroundProcessing nodes. "Failed to initialize StreamAPI: attempt 1 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null"

There is no external service for Kafka and Elasticsearh. Also was no connection details for timeout issue ( source & target ip , port) in logs after changing log levels for cluster and stream class.

Troubleshooting Steps Already Taken:

Is there any configuration for internal stream conf  ? What are the common causes for a TimeoutException during StreamAPI initialization on Search and BackgroundProcessing nodes?

 

*************************        Logs ***********************************************

Show More

Hi Everyone

After installed '23 successfully, we failed to start Pega as Stream node.I successfully addressed a Stream node issue by adding <env name="services/stream/provider" value="Internal"/> to prconfig.xml . Stream node is running.

However still are facing another issue on Search and BackgroundProcessing nodes. "Failed to initialize StreamAPI: attempt 1 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null"

There is no external service for Kafka and Elasticsearh. Also was no connection details for timeout issue ( source & target ip , port) in logs after changing log levels for cluster and stream class.

Troubleshooting Steps Already Taken:

Is there any configuration for internal stream conf  ? What are the common causes for a TimeoutException during StreamAPI initialization on Search and BackgroundProcessing nodes?

 

*************************        Logs ***********************************************

(.stream.StreamServiceValidator) ERROR    - Failed to initialize StreamAPI: attempt 1 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null         at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]         at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]         at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]         at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]         at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]         at java.lang.Thread.run(Thread.java:834) ~[?:?] 2023-12-28 15:25:55,508 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (.stream.StreamServiceValidator) ERROR    - Failed to initialize StreamAPI: attempt 2 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null         at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]         at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]         at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]         at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]         at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]         at java.lang.Thread.run(Thread.java:834) ~[?:?] 2023-12-28 15:26:20,510 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (.stream.StreamServiceValidator) ERROR    - Failed to initialize StreamAPI: attempt 3 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null         at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]         at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]         at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]         at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]         at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]         at java.lang.Thread.run(Thread.java:834) ~[?:?] 2023-12-28 15:26:30,246 [rver.Default-Pulse:2] [  STANDARD] [                    ] [                    ] (api.server.StreamServerService) DEBUG    - Finished executing stream service pulse operation 2023-12-28 15:26:45,512 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (.stream.StreamServiceValidator) ERROR    - Failed to initialize StreamAPI: attempt 4 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null         at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]         at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]         at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]         at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]         at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]         at java.lang.Thread.run(Thread.java:834) ~[?:?] 2023-12-28 15:27:10,514 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (.stream.StreamServiceValidator) ERROR    - Failed to initialize StreamAPI: attempt 5 from 6. Next retry in 20 seconds java.util.concurrent.TimeoutException: null         at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]         at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]         at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]         at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]         at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]         at java.lang.Thread.run(Thread.java:834) ~[?:?] 2023-12-28 15:27:30,324 [rver.Default-Pulse:2] [  STANDARD] [                    ] [                    ] (api.server.StreamServerService) DEBUG    - Finished executing stream service pulse operation 2023-12-28 15:27:35,515 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (.stream.StreamServiceValidator) ERROR    - Failed to initialize StreamAPI: attempt 6 from 6 java.util.concurrent.TimeoutException: null         at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]         at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]         at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]         at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]         at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]         at java.lang.Thread.run(Thread.java:834) ~[?:?] 2023-12-28 15:27:35,530 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (   external.async.IStartupTask) ERROR    - Error in the execution of Startup Task: [com.pega.dsm.dnode.api.server.StreamServiceInitializationTask] com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) Error in custom provider, com.pega.platform.stream.StreamServiceException: Failed to initialize StreamAPI. Can not connect to the stream provider.   at com.pega.platform.modules.internal.guice.StreamServiceModule.configure(StreamServiceModule.java:53)   at com.pega.platform.modules.internal.guice.StreamServiceModule.configure(StreamServiceModule.java:53)   while locating com.pega.fnx.stream.api.StreamAPI

1 error         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1025) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) ~[guice-4.0.jar:?]         at com.pega.platform.modules.internal.ModulesBridgeImpl.getStreamAPI(ModulesBridgeImpl.java:661) ~[modules-bridge.jar:?]         at com.pega.dsm.dnode.api.server.StreamServerService.getStreamAPI(StreamServerService.java:232) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.validateStreamAPI(StreamServiceInitializationTask.java:95) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.access$100(StreamServiceInitializationTask.java:32) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:82) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask$1.run(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.util.PrpcRunnable.execute(PrpcRunnable.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper$2.run(ServiceHelper.java:295) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContextInternal(ServiceHelper.java:288) ~[d-node.jar:?]         at com.pega.dsm.dnode.impl.prpc.service.ServiceHelper.executeInPrpcContext(ServiceHelper.java:150) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.initStreamServerServiceAndValidate(StreamServiceInitializationTask.java:77) ~[d-node.jar:?]         at com.pega.dsm.dnode.api.server.StreamServiceInitializationTask.runTask(StreamServiceInitializationTask.java:51) ~[d-node.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask$1.run(StartupTask.java:55) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1381) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1124) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:931) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:897) ~[prprivate-session.jar:?]         at com.pega.pegarules.session.internal.async.startup.StartupTask.run(StartupTask.java:48) ~[prprivate-session.jar:?]         at java.lang.Thread.run(Thread.java:834) ~[?:?] Caused by: com.pega.platform.stream.StreamServiceException: Failed to initialize StreamAPI. Can not connect to the stream provider.         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:86) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         ... 23 more Caused by: java.util.concurrent.TimeoutException         at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1888) ~[?:?]         at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2023) ~[?:?]         at com.pega.platform.stream.StreamServiceValidator.ping(StreamServiceValidator.java:91) ~[stream.jar:?]         at com.pega.platform.stream.StreamServiceValidator.validate(StreamServiceValidator.java:69) ~[stream.jar:?]         at com.pega.platform.stream.StreamService.getStreamAPI(StreamService.java:74) ~[stream.jar:?]         at com.pega.platform.modules.internal.guice.StreamServiceModule.createStreamAPI(StreamServiceModule.java:58) ~[modules-bridge.jar:?]         at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103) ~[guice-4.0.jar:?]         at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.0.jar:?]         at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145) ~[guice-4.0.jar:?]         at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092) ~[guice-4.0.jar:?]         at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012) ~[guice-4.0.jar:?]         ... 23 more 2023-12-28 15:27:35,531 [] [  STANDARD] [                    ] [                    ] (   external.async.IStartupTask) INFO     - All registered 'StartupTasks' (parallel cache loading) have finished. 2023-12-28 15:27:35,532 [] [  STANDARD] [                    ] [                    ] (   internal.mgmt.PREnvironment) ERROR    - com.pega.pegarules.pub.PRRuntimeException: Startup Task: [INITIALIZE_STREAM_SERVICE] has failed after retries or with error

Thank you in advance for any suggestions, advice, or shared experiences related to this issue.

Show Less
***Edited by Moderator Rupashree to add Capability tags***
***Edited by Moderator Marije to add Support Case Details INC-B417***
To see attachments, please log in.
Pega Platform '23
Pega Platform
Installation and Deployment
DX API
Java and Activities
Support Case Exists

Accepted Solution

Posted: 1 year ago
Updated: 1 year ago
Posted: 26 Jan 2024 10:50 EST
Updated: 30 Jan 2024 10:51 EST
Marije Schillern (MarijeSchillern)
MOD
Senior Knowledge Management Specialist
Pegasystems Inc.
GB

 @kasime17008587  please can you respond to the requests from GCS?   INC-B417 can be access via MSP

Analysis was that this is a straight forward issue as you're using an invalid PRCONFIG for external stream configuration. There was an issue with a previous version of our documentation which this incorrect PRCONFIG was provided. It was fixed few months ago but looks like you still have that version of instructions.

As the solution was provided which should address your configuration issue, we are planning to move this case to closure by tomorrow EOD If we don't hear back from you, so you can try the provided solution and re-open the case in case you still need assistance with the same issue.

Jan 23, 2024 07:32 PM

Current Status:

(logs from 1/14). Your startup process is failing to initialize StreamAPI because it’s looking for external stream connection properties, as shown below:

2024-01-14 12:15:42,659 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (prpc.service.ServiceDefinition) ERROR    - Unable to create DSM service DATA-DECISION-SERVICE-STREAMSERVER DEFAULT

Show More

 @kasime17008587  please can you respond to the requests from GCS?   INC-B417 can be access via MSP

Analysis was that this is a straight forward issue as you're using an invalid PRCONFIG for external stream configuration. There was an issue with a previous version of our documentation which this incorrect PRCONFIG was provided. It was fixed few months ago but looks like you still have that version of instructions.

As the solution was provided which should address your configuration issue, we are planning to move this case to closure by tomorrow EOD If we don't hear back from you, so you can try the provided solution and re-open the case in case you still need assistance with the same issue.

Jan 23, 2024 07:32 PM

Current Status:

(logs from 1/14). Your startup process is failing to initialize StreamAPI because it’s looking for external stream connection properties, as shown below:

2024-01-14 12:15:42,659 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (prpc.service.ServiceDefinition) ERROR    - Unable to create DSM service DATA-DECISION-SERVICE-STREAMSERVER DEFAULT

java.lang.reflect.InvocationTargetException: null […] Caused by: com.pega.platform.stream.StreamServiceException: Failed to configure StreamAPI provider […] Caused by: com.pega.platform.stream.StreamServiceException: 'services/stream/broker/url' isn't provided.

 

solution: Please try also providing the below PRCONFIG to the BackgroundProcessing node and see if that addresses the issue.

If not, please provide fresh startup logs from Stream and BackgroundProcessing nodes so I can review the startup history.

<env name="services/stream/provider" value="Internal"/>

This PRCONFIG in theory would force the JVM to look for an internal stream node in the same cluster. Please confirm if you have tried this already or not.

If it doesn’t work, please try to change the node classification for the backgroundprocessing node and add Stream node type to it and see if it joins the same cluster as the other stream node. If it doesn’t identify the other node, it’s most likely a hazelcast/cluster management configuration issue or the nodes aren’t able to communicate through network.

 

Next Steps:

  • Client – to try the above suggestion and confirm back. If it doesn’t work, client to provide fresh startup logs for both stream and backgroundprocessing node so GCS can review.

 

 

***Edited by Moderator Marije to add update a week later ***

No response so we will assume the issue is now resolved with the above steps. 

****************

 

Show Less
To see attachments, please log in.
Posted: 1 year ago
Updated: 1 year ago
Posted: 1 Jan 2024 8:23 EST
Updated: 1 Jan 2024 8:49 EST
Saikishore Sanagapalli (Kishore Sanagapalli)
LTIMindtree
Senior Specialist - Pega Containerization Practitioner
LTIMindtree
SA

@kasime17008587 

Hi, May I know your inputs on the below queries first?

1. Was all the nodes working before prior to Pega Infinity 23 installation?

2. Pega Infinity 23 is the first installation in On-Prem or was upgraded from any other version? if upgraded from different version? which one is it?

3. Since it's On-prem installation, May I know your environment details? and tools being used?

4. Did you check urandom parameter is configured?

5. did you make sure Pega-Engine • prconfig/services/stream/provider/default [ Value: ExternalKafka] exist by default when you install Pega Infinity'23. 

To see attachments, please log in.
Posted: 1 year ago
Posted: 2 Jan 2024 6:20 EST
kasim ergun (kasime17008587)
sirar by stc

sirar by stc
SA

@Kishore Sanagapalli Thanks for replying. Kindly check the info.

 

1. It is the first start execution for all nodes.

2. Pega Infinity 23 is the first installation in On-Prem 

3.  Apache Tomcat 9 Server and Ms sql 15 DB

4. Did you check urandom parameter is configured?  NO Please more details

5. Stream type is internal and applied in prconfig.xml (  On-premise '23 Pega platform does not start with Tomcat | Support Center applied to prconfig.xml)

To see attachments, please log in.
Posted: 1 year ago
Posted: 2 Jan 2024 7:00 EST
Saikishore Sanagapalli (Kishore Sanagapalli)
LTIMindtree
Senior Specialist - Pega Containerization Practitioner
LTIMindtree
SA

@kasime17008587

Thanks for the details.

about urandom: -Djava.security.egd=file:///dev/urandom is one of the JVM argument plays major role in the genrating requestors for Pega. Since It's on-prem, You need to check this parameter is configured or not in your middleware application at the JVM arguments definition section.

If it's not there, please add it and restart the JVM's and check once.

If you need more help whether urandom is configured ot not,

Issue the below command from your linux terminal:

ps -ef | grep java

Note: Let me know the outcome

To see attachments, please log in.
Posted: 1 year ago
Posted: 3 Jan 2024 4:48 EST
Anthony Gourtay (Anthony_Gourtay)
Proximus
Expert Engineer - PEGA Application Management
Proximus
BE

@kasime17008587

Hello,

Is it a fresh installation or an upgrade from a previous version? which version in case of.

Have you specified port for your stream service in prconfig?

Have you well defined node classification for Stream, Search & Backgroundprocessing?

What's shown in Decisioning > infrastructure > Services > Stream?

Is it empty? or you've got old ref ready to be decomisionned  ?

Do you see your node?

Have you tried a clean restart with pegatemp cleared and all stream tables emptied?

Thank you

Regards

Anthony

To see attachments, please log in.
Posted: 1 year ago
Posted: 11 Jan 2024 1:38 EST
kasim ergun (kasime17008587)
sirar by stc

sirar by stc
SA

Hi @Anthony_Gourtay

Thanks your supports in advance. still not fixed it.  I raised SR regarding the issue. I share the solution here once fixed 

 

Please find my answer.

Is it a fresh installation or an upgrade from a previous version? which version in case of.  /// New Installation

Have you specified port for your stream service in prconfig?  /// no override regarding ports

Have you well defined node classification for Stream, Search & Backgroundprocessing? /// yes

What's shown in Decisioning > infrastructure > Services > Stream? /// Only stream nodes is running.

Is it empty? or you've got old ref ready to be decomisionned  ?/// empty.

Do you see your node? /// yes . under decisioning landing and on Admin studio

Have you tried a clean restart with pegatemp cleared and all stream tables emptied?  /// yes.

To see attachments, please log in.
Posted: 1 year ago
Updated: 1 year ago
Posted: 19 Jan 2024 6:25 EST
Updated: 19 Jan 2024 6:26 EST
Marije Schillern (MarijeSchillern)
MOD
Senior Knowledge Management Specialist
Pegasystems Inc.
GB

@kasime17008587  Please can you confirm that your open ticket is INC-B417 ( Background node does not start on On-premise Pega '23)  that was logged 4th January?

Our GCS team have been waiting for a response from you since the 15th January regarding the below ticket update: 

-----------------------------------------------

Current Status:

By looking at the stack trace, the background processing node is trying to start the StreamAPI module and perform validation of the stream service by verifying if a stream node is up in the same cluster, the stream validation is enabled in all nodes by default.

The exception is a straight timeout, which means the stream node is possibly not reachable from the background processing node perspective, as it’s not identifying the kafka server and a stream server in NORMAL status to perform the stream validation.

---> Can you please ensure that both servers can reach each other through the local network and also check if there is any firewall rule possibly blocking this connection?

--->  can you please provide logs from the stream node which was already running when you tried to bring you the background processing node, so I can review the initialization for that node as well.

-----------------------------------------------

Next Steps:

Show More

@kasime17008587  Please can you confirm that your open ticket is INC-B417 ( Background node does not start on On-premise Pega '23)  that was logged 4th January?

Our GCS team have been waiting for a response from you since the 15th January regarding the below ticket update: 

-----------------------------------------------

Current Status:

By looking at the stack trace, the background processing node is trying to start the StreamAPI module and perform validation of the stream service by verifying if a stream node is up in the same cluster, the stream validation is enabled in all nodes by default.

The exception is a straight timeout, which means the stream node is possibly not reachable from the background processing node perspective, as it’s not identifying the kafka server and a stream server in NORMAL status to perform the stream validation.

---> Can you please ensure that both servers can reach each other through the local network and also check if there is any firewall rule possibly blocking this connection?

--->  can you please provide logs from the stream node which was already running when you tried to bring you the background processing node, so I can review the initialization for that node as well.

-----------------------------------------------

Next Steps:

Please get in touch with GCS to confirm on above items and provide logs from the stream node which should be running in the same Pega cluster. 

Show Less
To see attachments, please log in.

Accepted Solution

Posted: 1 year ago
Updated: 1 year ago
Posted: 26 Jan 2024 10:50 EST
Updated: 30 Jan 2024 10:51 EST
Marije Schillern (MarijeSchillern)
MOD
Senior Knowledge Management Specialist
Pegasystems Inc.
GB

 @kasime17008587  please can you respond to the requests from GCS?   INC-B417 can be access via MSP

Analysis was that this is a straight forward issue as you're using an invalid PRCONFIG for external stream configuration. There was an issue with a previous version of our documentation which this incorrect PRCONFIG was provided. It was fixed few months ago but looks like you still have that version of instructions.

As the solution was provided which should address your configuration issue, we are planning to move this case to closure by tomorrow EOD If we don't hear back from you, so you can try the provided solution and re-open the case in case you still need assistance with the same issue.

Jan 23, 2024 07:32 PM

Current Status:

(logs from 1/14). Your startup process is failing to initialize StreamAPI because it’s looking for external stream connection properties, as shown below:

2024-01-14 12:15:42,659 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (prpc.service.ServiceDefinition) ERROR    - Unable to create DSM service DATA-DECISION-SERVICE-STREAMSERVER DEFAULT

Show More

 @kasime17008587  please can you respond to the requests from GCS?   INC-B417 can be access via MSP

Analysis was that this is a straight forward issue as you're using an invalid PRCONFIG for external stream configuration. There was an issue with a previous version of our documentation which this incorrect PRCONFIG was provided. It was fixed few months ago but looks like you still have that version of instructions.

As the solution was provided which should address your configuration issue, we are planning to move this case to closure by tomorrow EOD If we don't hear back from you, so you can try the provided solution and re-open the case in case you still need assistance with the same issue.

Jan 23, 2024 07:32 PM

Current Status:

(logs from 1/14). Your startup process is failing to initialize StreamAPI because it’s looking for external stream connection properties, as shown below:

2024-01-14 12:15:42,659 [ALIZE_STREAM_SERVICE] [  STANDARD] [                    ] [                    ] (prpc.service.ServiceDefinition) ERROR    - Unable to create DSM service DATA-DECISION-SERVICE-STREAMSERVER DEFAULT

java.lang.reflect.InvocationTargetException: null […] Caused by: com.pega.platform.stream.StreamServiceException: Failed to configure StreamAPI provider […] Caused by: com.pega.platform.stream.StreamServiceException: 'services/stream/broker/url' isn't provided.

 

solution: Please try also providing the below PRCONFIG to the BackgroundProcessing node and see if that addresses the issue.

If not, please provide fresh startup logs from Stream and BackgroundProcessing nodes so I can review the startup history.

<env name="services/stream/provider" value="Internal"/>

This PRCONFIG in theory would force the JVM to look for an internal stream node in the same cluster. Please confirm if you have tried this already or not.

If it doesn’t work, please try to change the node classification for the backgroundprocessing node and add Stream node type to it and see if it joins the same cluster as the other stream node. If it doesn’t identify the other node, it’s most likely a hazelcast/cluster management configuration issue or the nodes aren’t able to communicate through network.

 

Next Steps:

  • Client – to try the above suggestion and confirm back. If it doesn’t work, client to provide fresh startup logs for both stream and backgroundprocessing node so GCS can review.

 

 

***Edited by Moderator Marije to add update a week later ***

No response so we will assume the issue is now resolved with the above steps. 

****************

 

Show Less
To see attachments, please log in.

We'd prefer it if you saw us at our best.

Pega Collaboration Center has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice