Skip to main content

This content is closed to future replies and is no longer being maintained or updated.
Links may no longer function. If you have a similar request, please write a new post.

Question

2
Replies
239
Views
 Miguel Angel Calderon Bravo (Miguel Calderon) Plexus Tech Senior System Architect (CSSA)
Plexus Tech
ES
Miguel Calderon Member since 2018 13 posts
Plexus Tech
Posted: August 28, 2019
Last activity: August 29, 2019
Posted: 28 Aug 2019 16:01 EDT
Last activity: 29 Aug 2019 7:07 EDT
Closed

Partition keys in Stream Data Set rules

Report

Hi,
I created a Data Set to get messages from a kafka queue that has 2 partitions.
How do I assign value to the property specified as Partition Key?

Attached captures with the Data Set definition.

Thanks.

Dev/Designer Studio Data Integration
Posted: 3 years ago
Posted: 29 Aug 2019 5:39 EDT
Nishitha Chintala (chinn2) PEGA Senior Cloud Development Engineer, Management Plane Services
Pegasystems Inc.
IN
Report

Hi Miguel Calderon,

What is the pega version that you are using?

From pega 7.4 ,by default number of partitions for kafka and stream dataset are 20(no configuration is needed). If you want to change this default number of partitions, go to kafka folder (in application server folder) and open connect-distributed.properties file and change the value of num.partitions.

By configuring partitioning you can ensure that related records are sent to the same partition. If no partition keys are set, the Kafka data set randomly assigns records to partitions.

Please go through below link for more information related to kafka data set creation.

Show More

Hi Miguel Calderon,

What is the pega version that you are using?

From pega 7.4 ,by default number of partitions for kafka and stream dataset are 20(no configuration is needed). If you want to change this default number of partitions, go to kafka folder (in application server folder) and open connect-distributed.properties file and change the value of num.partitions.

By configuring partitioning you can ensure that related records are sent to the same partition. If no partition keys are set, the Kafka data set randomly assigns records to partitions.

Please go through below link for more information related to kafka data set creation.

https://community.pega.com/sites/default/files/help_v74/procomhelpmain.htm

Generally to set the value of the property which is used as partition key for the dataset, you need to send the value in request itself.

Let's say i am using Age property as my partition key for stream data set, I need to send Age property value along with the required properties while sending the request to stream data set.

Hope it helps.Let me know if you have any queries. 


Show Less
Posted: 3 years ago
Posted: 29 Aug 2019 7:07 EDT
Miguel Angel Calderon Bravo (Miguel Calderon) Plexus Tech Senior System Architect (CSSA)
Plexus Tech
ES
Report

Hi chinn2,
My pega version is 7.4.

Thank you for the information.

Regards

We'd prefer it if you saw us at our best.

Pega Collaboration Center has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice