What happens if there is no Stream node/service running in a cluster?
Hi,
Greetings. Hope you are well! I wanted to know what happens if there is no stream node/service is running in a cluster? Our cluster with WebUser, BackgroundProcessing, Search is all up and running and i even do not get any message saying "Stream service is not running in the cluster". Can someone please let me know the implications of not running a stream node in a cluster?
Regards,
Bharat
Lantiqx Limited
IN
NCS Pte. Ltd
SG
Hi @KOMARINA: The impact is for the queue processors.
If a stream node is not available when the system queues a message, the system saves the message in a database, and then pushes the message to Kafka when the stream node becomes available again.
Thanks.
Updated: 13 Dec 2022 8:51 EST
LTIMindtree
SA
Are you talking about the Stream Node Impact in On-Prem enviornment or Cloud environment?
Common Sysmptoms in On-Prem & Cloud:
1. Once Stream Node is stopped running / Un-reachanble state, Application Will become a Skeleton one. Of-Course, You will not face any issue while logging in etc..
2. Queue Processors and Kafka Messeges Streaming will be haulted unless you restart the Stream Node again.
3. Stream Node issues will come when the connection between Stream Service and Database is lost even for very less seconds.
Cloud Enviornment:
In any cloud environment, Stream Services will try to re-establish the connection and the Services will become NORMAL, In most of the occasaions. Even when there's a chance that Stream Service did not come back to NORMAL, You can perform the Kafka folder clean up and the Service will come back to NORMAL with in less time, unlike in On-Prem. Once the Service is NORMAL, Kafka Messages / Stream gets streamlined automatically, In most of the cases.
On-Prem Environment:
It's usual that Stream Service Breakage even when you're maintaining GOLD Level standard of Application High Availability. In On-Prem, When the issue occurs, Each time you'll need to perform Stream Node / JVM restart and if the Node is not joinging back in the Cluster somoothly, You may have to bring down the whole application again to stablize it.
Are you talking about the Stream Node Impact in On-Prem enviornment or Cloud environment?
Common Sysmptoms in On-Prem & Cloud:
1. Once Stream Node is stopped running / Un-reachanble state, Application Will become a Skeleton one. Of-Course, You will not face any issue while logging in etc..
2. Queue Processors and Kafka Messeges Streaming will be haulted unless you restart the Stream Node again.
3. Stream Node issues will come when the connection between Stream Service and Database is lost even for very less seconds.
Cloud Enviornment:
In any cloud environment, Stream Services will try to re-establish the connection and the Services will become NORMAL, In most of the occasaions. Even when there's a chance that Stream Service did not come back to NORMAL, You can perform the Kafka folder clean up and the Service will come back to NORMAL with in less time, unlike in On-Prem. Once the Service is NORMAL, Kafka Messages / Stream gets streamlined automatically, In most of the cases.
On-Prem Environment:
It's usual that Stream Service Breakage even when you're maintaining GOLD Level standard of Application High Availability. In On-Prem, When the issue occurs, Each time you'll need to perform Stream Node / JVM restart and if the Node is not joinging back in the Cluster somoothly, You may have to bring down the whole application again to stablize it.
Sometimes, when you perform the restart of Stream JVM, You'll need to perform the Kafka Folder clean up manually even there are issues still in-tact while joining back in the cluster again. Atleast, 45min of duration is ecpected for Stream JVM during the toss. 45min is a very minimal Duration of Recovery (Post maintaining the JVM with Utomost performance is Tuned in both Middleware and Application Layer).
Note: You may mark my input as accepted solution, if my inputs clarifies your query.
@KOMARINA ,
Pega Uses Kafka as a streaming service
From 8 version onwards queue processors were introduced which internally used the kafka integration /kafka folders to process the requests . And this KAFKA integration will be used as a streaming service.
If you observe any issues related to streaming service joining failed it means our kafka is not working proper . and we need to add some configuration changes to start it proper. Please Check below link
https://docs-previous.pega.com/decision-management-reference-materials/kafka-streaming-service#:~:text=Kafka%20is%20a%20high%2Dthroughput,of%20event%20and%20message%20queues.