Hi,

We are using Pega 8.1.5 version which is configured on websphere 8.5.5 and Oracle 12.2.

Recent after the restart in our Production environment, we can’t get a STREAM connected due to below error. Any suggestion?

Node ID

0f5a02a02ad496b149108a84f3f4872e

Error message

StreamServiceException: Unable to start kafka broker, broker hangs in state: RecoveringFromUncleanShutdown
[2019-09-11 12:05:12,819] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2019-09-11 12:05:13,435] INFO starting (kafka.server.KafkaServer)
[2019-09-11 12:05:13,436] INFO Connecting to zookeeper on Proprietary information hidden:2181 (kafka.server.KafkaServer)
[2019-09-11 12:05:13,464] INFO [ZooKeeperClient] Initializing a new session to Proprietary information hidden:2181. (kafka.zookeeper.ZooKeeperClient)
[2019-09-11 12:05:13,481] INFO Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,481] INFO Client environment:host.name= Proprietary information hidden (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,481] INFO Client environment:java.version=1.8.0_191 (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,481] INFO Client environment:java.vendor=IBM Corporation (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,481] INFO Client environment:java.home=/sb/sys/was/L1/appserver/java/jre (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,481] INFO Client environment:java.class.path=/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/aopalliance-repackaged-2.5.0-b32.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/log4j-1.2.17.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/kafka-tools-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-http-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/hk2-api-2.5.0-b32.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/javax.inject-1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jackson-module-jaxb-annotations-2.9.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/guava-20.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/argparse4j-0.7.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jersey-guava-2.25.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-servlet-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/connect-file-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/rocksdbjni-5.7.3.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-security-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/javax.ws.rs-api-2.0.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-servlets-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/maven-artifact-3.5.2.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jersey-container-servlet-core-2.25.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/snappy-java- Proprietary information hidden.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/kafka-streams-examples-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/connect-transforms-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/osgi-resource-locator-1.0.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/slf4j-api-1.7.25.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/hk2-locator-2.5.0-b32.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/commons-lang3-3.5.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jersey-client-2.25.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/hk2-utils-2.5.0-b32.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/plexus-utils-3.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jackson-core-2.9.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/javax.annotation-api-1.2.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jackson-databind-2.9.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/kafka_2.11- Proprietary information hidden.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-util-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jackson-annotations-2.9.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/zkclient-0.10.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jersey-server-2.25.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/kafka-clients- Proprietary information hidden.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/zookeeper-3.4.10.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/scala-reflect-2.11.12.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/connect-api-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jersey-media-jaxb-2.25.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/connect-json-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jackson-jaxrs-json-provider-2.9.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/lz4-java-1.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-continuation-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/javassist-3.20.0-GA.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/slf4j-log4j12-1.7.25.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/kafka-log4j-appender-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-io-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/validation-api-1.1.0.Final.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jopt-simple-5.0.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jackson-jaxrs-base-2.9.4.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/javassist-3.21.0-GA.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/javax.servlet-api-3.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/kafka-streams-test-utils-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jersey-common-2.25.1.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-client-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jetty-server-9.2.24.v20180105.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/kafka-streams-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/javax.inject-2.5.0-b32.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/reflections-0.9.11.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/scala-library-2.11.12.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/connect-runtime-1.1.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/metrics-core-2.2.0.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/scala-logging_2.11-3.7.2.jar:/sb/env/app/S10/temp/kafka- Proprietary information hidden/libs/jersey-container-servlet-2.25.1.jar (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:java.library.path=/sb/sys/was/L1/appserver/java/jre/lib/amd64/compressedrefs:/sb/sys/was/L1/appserver/java/jre/lib/amd64:/sb/sys/was/L1/appserver/lib/native/linux/x86_64/:/sb/sys/was/L1/appserver/bin:/sb/sys/was/L1/appserver/nulldllsdir:/usr/lib64:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:java.compiler= (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:os.version=3.10.0-957.21.3.el7.x86_64 (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:user.name=p986wasuser (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:user.home=/export/home/p986wasuser (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,482] INFO Client environment:user.dir=/sb/env/app/S10/temp/kafka- Proprietary information hidden/bin (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,485] INFO Initiating client connection, connectString= Proprietary information hidden:2181 sessionTimeout=30000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@60389438 (org.apache.zookeeper.ZooKeeper)
[2019-09-11 12:05:13,514] INFO Opening socket connection to server Proprietary information hidden/ Proprietary information hidden:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2019-09-11 12:05:13,516] INFO [ZooKeeperClient] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2019-09-11 12:05:13,519] INFO Socket connection established to Proprietary information hidden/ Proprietary information hidden:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2019-09-11 12:05:13,667] INFO Session establishment complete on server Proprietary information hidden/ Proprietary information hidden:2181, sessionid = 0xffffffffb7a90001, negotiated timeout = 30000 (org.apache.zookeeper.ClientCnxn)
[2019-09-11 12:05:13,674] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.ZooKeeperClient)
[2019-09-11 12:05:14,287] INFO Cluster ID = 6LpSc_cWTYig405gRlSBIA (kafka.server.KafkaServer)
[2019-09-11 12:05:14,374] INFO KafkaConfig values:

***Moderator Edit-Vidyaranjan: Updated Platform Capability***