Question
Maantic Inc
Maantic Inc
US
Maantic Inc
Posted: Apr 28, 2022
Last activity: Apr 28, 2022
Last activity: 28 Apr 2022 21:48 EDT
Closed
Partition key in Kafka data set is not working as expected
Hi
DataFlow is configured to read the data from the report definition and post the data in the Kafka topic.
Kafka topic has five partitions and messages have a partition key.
When messages are published through dataset, messages going to only a few partitions.
Actual result
| Message | Partition | Partition_Id |
| Message 1 | Partition 1 | P1 |
| Message 2 | Partition 1 | P2 |
| Message 3 | Partition 1 | P3 |
| Message 4 | Partition 3 | P4 |
| Message 5 | Partition 4 | P5 |
Expected result
| Message | Partition | Partition_Id |
| Message 1 | Partition 0 | P1 |
| Message 2 | Partition 1 | P2 |
| Message 3 | Partition 2 | P3 |
| Message 4 | Partition 3 | P4 |
| Message 5 | Partition 4 | P5 |
DataSet Config
Show More
Hi
DataFlow is configured to read the data from the report definition and post the data in the Kafka topic.
Kafka topic has five partitions and messages have a partition key.
When messages are published through dataset, messages going to only a few partitions.
Actual result
| Message | Partition | Partition_Id |
| Message 1 | Partition 1 | P1 |
| Message 2 | Partition 1 | P2 |
| Message 3 | Partition 1 | P3 |
| Message 4 | Partition 3 | P4 |
| Message 5 | Partition 4 | P5 |
Expected result
| Message | Partition | Partition_Id |
| Message 1 | Partition 0 | P1 |
| Message 2 | Partition 1 | P2 |
| Message 3 | Partition 2 | P3 |
| Message 4 | Partition 3 | P4 |
| Message 5 | Partition 4 | P5 |
DataSet Config
Show Less