Kafka via Avro Schema
Hi All,
We use Kafka via Avro schema format; the connectivity in the Kafka connection and data set is successful, but when we run the data set, the Kafka table has the following exceptions:
Caused by: com.pega.dsm.dnode.api.ExceptionWithInputRecord: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 434<CR>
Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 434<CR>
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target<CR>
I see a similar question in the Support Center:
https://support.pega.com/question/kafka-data-avro-schema
and the solution is: The certificate needs to be uploaded to the app server truststore JKS file.
Does anyone know where to set the certificate for the schema registry server, or is there any suggestion to resolve this issue?
Thank you.
***Edited by Moderator Marije to add BUG fix***
Accepted Solution
Updated: 28 Feb 2024 6:23 EST
Pegasystems Inc.
GB
@AnggunMD In Pega Infinity, you can set up a secure connection to an Avro schema registry by using SSL. You need to create Truststore and Keystore data instances by uploading the files in the JKS format. The Truststore file holds the certificates from the Certificate Authority of the schema registry server. Pega Platform uses the Truststore to verify the certificate presented by the server in an SSL connection. The Keystore file contains the client certificates and private keys trusted by the Certificate Authority of the schema registry server. You can configure these settings in the 'Security settings' section when editing the Avro Schema.
⚠ This is a GenAI-powered tool. All generated answers require validation against the provided references.
Configuring Avro schema for Kafka data set
Configuring an Avro schema for a Kafka data set > Defining an SSL connection to an Avro schema registry
More info:
Pega started supporting Avro message format starting from Pega 8.7 version.
But as part of Avro Schema rule which is used for connecting to Schema Registry, Pega OOTB does not provide configuration option to use SSL. Because of this connection to schema registry using SSL is not possible OOTB.
@AnggunMD In Pega Infinity, you can set up a secure connection to an Avro schema registry by using SSL. You need to create Truststore and Keystore data instances by uploading the files in the JKS format. The Truststore file holds the certificates from the Certificate Authority of the schema registry server. Pega Platform uses the Truststore to verify the certificate presented by the server in an SSL connection. The Keystore file contains the client certificates and private keys trusted by the Certificate Authority of the schema registry server. You can configure these settings in the 'Security settings' section when editing the Avro Schema.
⚠ This is a GenAI-powered tool. All generated answers require validation against the provided references.
Configuring Avro schema for Kafka data set
Configuring an Avro schema for a Kafka data set > Defining an SSL connection to an Avro schema registry
More info:
Pega started supporting Avro message format starting from Pega 8.7 version.
But as part of Avro Schema rule which is used for connecting to Schema Registry, Pega OOTB does not provide configuration option to use SSL. Because of this connection to schema registry using SSL is not possible OOTB.
Please see the Resolved Issues documentation for the earliest patch versions where this is fixed. (BUG-804631/BUG-806753)
Pega Platform 8.7.6 Patch Resolved Issues