Externalization of kafka - IBM Event Streams
Hi All,
As per this, it is mentioned that "you can buy a Kafka license from a provider, for example Confluent". Does PEGA supports IBM event streams as external kafka for its stream service?
This information is not available in platform support guide too. Is there a list which we can refer to know the compactable kafka providers which PEGS supports.
Thanks.
Accepted Solution
Updated: 29 Sep 2023 0:37 EDT
Pegasystems Inc.
NL
@ArulDevan :Thanks for the question: Currently we don't have list of compatible Kafka providers that you can use to connect with pega platform as external stream. But I can answer it in a little bit different format.
Essentially any kafka provider should support following 2 main points
- Supports/be compatible with Kafka protocol
- Supports any of the following authentication mechanims to connect to kafka
- SASL using JAAS
- SASL/PLAIN
- SASL/SCRAM
Reference Doc: https://docs.pega.com/bundle/platform/page/platform/deployment/externalization-of-services/externalize-kafka-in-your-deployment.html
Now coming to IBM Event streams, it satisfies both requirements mentioned above as it
- supports Kafka Protocol
- supports connection via SASL PLAIN
https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-connecting https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-kafka_using#kafka_api_client
Hope this helps.
Updated: 19 Sep 2023 9:00 EDT
Pegasystems Inc.
GB
Hi @ArulDevan I was also not able to find these details in the documentation.
Would you be able to submit a documentation feedback ticket against the External Kafka in your deployment article and suggest that they include these details there?
Can you confirm that the Support Document Third-party externalized services Deployment Changes FAQs also does not clarify this point?
As the article has been left open for response, feel free to hit 'Reply' on the main content in order to ask your question directly to the author who wrote the support document.
NCS Pte. Ltd
SG
Hi @MarijeSchillern: Thanks for the guidance. Have raised a feedback in the documentation and also raised a query in other FAQ support doc.
Thanks.
Ford Motor Company
US
@ArulDevan Please do share the update here as well.
To my knowledge, IBM Event Streams is not listed as official supported streaming service, however, it appears, it might work as long as IBM Event Streams suports API Key or User ID/Password Authentication.
NCS Pte. Ltd
SG
Hi @MarijeSchillern and moderator team: Can you please help to get support from the product team on this?
Thanks.
Pegasystems Inc.
GB
@ArulDevan I am already tracking your requests and am locating the SME's for you. We will respond asap.
Pegasystems Inc.
GB
@ArulDevan our SME posted a response via the Support Document: Third-party externalized services Deployment Changes FAQs
"This doc https://docs.pega.com/bundle/platform-88/page/platform/deployment/exter… provides details on setting up Elasticsearch.
Services like kibana/logstash/beats are all external to Pega and are up to the client to choose and deploy as needed."
Let me know which questions are still outstanding and I'll ask them to post a response here.
NCS Pte. Ltd
SG
Hi @MarijeSchillern: Thanks for tracking this. Noticed that the reply in the other thread is for elastic search. The query on Kafka is still open.
Thanks.
Updated: 27 Sep 2023 5:15 EDT
Pegasystems Inc.
GB
@ArulDevan I expect the answer is that if IBM Event Streams are fully compatible with Kafka, it should work.
@malaa1 would you be able to provide some input on this Kafka question?
" Is there a list which we can refer to know the compatible kafka providers which PEGS supports?"
Accepted Solution
Updated: 29 Sep 2023 0:37 EDT
Pegasystems Inc.
NL
@ArulDevan :Thanks for the question: Currently we don't have list of compatible Kafka providers that you can use to connect with pega platform as external stream. But I can answer it in a little bit different format.
Essentially any kafka provider should support following 2 main points
- Supports/be compatible with Kafka protocol
- Supports any of the following authentication mechanims to connect to kafka
- SASL using JAAS
- SASL/PLAIN
- SASL/SCRAM
Reference Doc: https://docs.pega.com/bundle/platform/page/platform/deployment/externalization-of-services/externalize-kafka-in-your-deployment.html
Now coming to IBM Event streams, it satisfies both requirements mentioned above as it
- supports Kafka Protocol
- supports connection via SASL PLAIN
https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-connecting https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-kafka_using#kafka_api_client
Hope this helps.