@JohnPaulRaja,C in your question you have confirmed that Pega does not support this process.

I'm not clear on why you are asking for help in implementing an unsupported configuration....

You have not clarified what Pega version you are on but I believe the product is working as designed.

Understanding Queue Processors

Queue Processor FAQ

In Pega Platform, a partition key is used in a Database Table data set to distribute read operations into partitions across the nodes. However, the Pega Platform doesn’t support processing messages in an ordered sequence in the context of Queue Processors. Queue Processors are designed for processing massive amounts of messages asynchronously. To enhance the performance of your queue processor rules, you can scale out by increasing the number of processing nodes or scale up by increasing the number of threads per node, up to 6 threads per cluster. If you need more threads (partitions) per cluster, you can increase the number of partitions. To change the number of partitions, you can use the pxAlterStreamPartitions activity. In the Run Activity: AlterStreamPartitions dialog box, enter the name of the queue processor in uppercase letters in the stream field and enter the desired number of partitions in the partitions field. Please note that the maximum number of Kafka partitions is 6 for one Pega Platform cluster, which means that the maximum number of queue processor threads that can process messages across all nodes in a cluster is 6.

Example of steps to reproduce

1. Queue processor is turned off.

2. Created 3 different Queue Items related to Same Work Object. ( Enrollment , In-Progress, Closed )

3. In Schedule State all 3 Items are in Sequence

4. After Specific scheduled time interval moved to Ready to Process

5. Queue Processor is started.

6. Queue Items are processed but not in Sequence. ( Closed , In progress, Enrollment )

When you create a topic, it creates multiple partition by default. The messages are posted to QP in oder but kafka partitioner uses hash on partition key to distribute records across the partitions. So, record going into same partition will be in order but if you go to different partition then we can't guarantee order.

Below information I could find about your issue for a client who asked about this on a 7.4 implementation::

Producer needs to use a partition key to ensure that data for same partition key goes to the same partition. Partition key is required for both producer and consumer. Specify the partition key in Kafka dataset for parallelization in processing.

To restrict messages to partition zero, the producer has to ensure that they are sending messages with same partition key always. If there is a requirement to consume messages only from specific partition then the logic needs to be handled at the data flow level. Pega's Kafka dataset will read all messages from the topic. But in Data Flow level filter conditions can be used in different branches to implement processing logic for a specific partition

If producer sends data to a single then distributed processing on Pega's side can't be achieved as data will always be read sequentially. Producer needs to choose a partition key such that it is able to distribute data across partitions and also send associated data to single partition. At data flow level filter can be used to only filter out records for required partition and eliminate the records from other partitions.

The offset is flushed or persisted to the database based on the snapshot interval configured under data flow resiliency. Pega doesn't commit offset back to Kafka after consuming a message.

We can see if someone on the forum can provide you with the help that you need for your particular scenario.