I am configuring an externally managed stream server on kafka cluster 2.6.0, based on Confluent, for Pega CDH
Connection is working fine and the Stream Service is reported as Normal in the Decisioning->Infrastructure->Services->Stream Window.
Unfortunately, I get a lot of error over two issues:
1) For customer requirement, I need to set a prefix to the consumer groups and found no documentation about how to do it: the application tries to create the consumer groups df_run_metrics_subscription and vbd_insert_subscription and I can't find anywhere where they are defined
2) The customer reports that Pega tries to use the outdated API Cluster -> Create instead of Topic -> Create, which is mandatory. The new API is described in Kafka KIP-277
Can anyone help me on the topics above? I have been looking in the Collaboration center for two days: found a lot of useful stuff, but nothing about that.
Can you please elaborate further about your requirement?
I am trying to understand the following:-
- Which consumer and consumer groups are you referring here?
- As you are maintaining an external Kafka who are you referring to only Pega side consumers in this case ?
- Are you referring to custom real time data flows sourced by Kafka/Stream datasets as consumers are you trying to group?
- It seems Pega does not use the concept of consumer group , as it keeps track of its position in the topic via the database rather than relying on Kafka. Also multiple consumer group creation is not supported.