Hi Melissa,

Apache Kafka is a distributed streaming service used by the Pega Decisioning applications until it is included in Pega platform starting from 8.x to support existing & new core platform features. 

At least one Stream nodeType is required for Pega 8.x application to run stream service under the hood. Refer to below discussion for insights on Stream service and relation with Apache Kafka - https://collaborate.pega.com/question/need-details-kafka-which-comes-pega-8

We don't need to manage the frequency of Apache Kafka as Platform takes care of initializing Kafka stream process and the creation of required directories. 

The installation guide of any 8.x version suggests having stream service enabled by nodeType. Disabling the Apache Kafka stream services can impact Queue Processors, Search and unsure if JVM can be started cleanly. 

Let me know if this helps!

Hi Thorm1,

Please find the point for the requested Questions.

1.What does Kafka do?

Kafka is a streaming service which comes inbuilt from Pega 8.x version . It is used for stream processing. It is similar to message Queue which stores the information in various Topics and Partitions.

2.Is Kafka required to run in our Pega 8.2 environment?  If yes, what frequency should it be run?

There is no compulsory to run Kafka in the environment. But to leverage the latest features that Pega is providing it is recommended to use Kafka. For example in the previous version of Pega there is a concept of Agents which are used for processing the items without any human intervention. Similar to Scheduled Jobs. Pega came up with Job Scheduler and Queue Processor for fast processing of items and enhanced some other functionalities. If Kafka Streaming is not enabled these kind of functionalities will not be useful. 

3.What is the impact of Kafka not running

Points specified in 2nd Question.

Regards,

Chaitanya.