Question
Merkle. Inc.
GB
Last activity: 12 May 2020 3:54 EDT
Queue Processeing from 8.3
Queue Processor uses Kafka queues for immediate queues and not DB Queue table.
Delayed Queues are stored in pr_sys_delayed_queue table.
Broken items are sotred in pr_sys_msg_qp_brokenitems
is this case, is there any need for customized class(or table) for Queues, if Queue items are large in number????
***Edited by Moderator Marissa to update Content Type from Discussion to Question***
Updated: 11 May 2020 19:55 EDT
Pegasystems Inc.
US
Please refer to below Queue processor related questions.
https://community.pega.com/knowledgebase/articles/system-administration/queue-processor-faq#10
Queued items to Queue Processors are not stored in Pega DB Queue table. Items information is directly pushed to Kafka and while it is processing it will retrieve the information from Kafka message Queue.
The maximum number of Kafka partitions is 20 for one Pega Platform cluster, which means that the maximum number of threads that can process messages across all nodes in a cluster is 20.
Pegasystems Inc.
US
I assume you are worried about performance when those tables are getting big. I checked the indices on pr_sys_msg_qp_brokenitmes, they are there including the column pzqueueprocessorname (which has an associated queue class). In theory we should be fine. Now the question is: are you already experiencing latency in your environments or is this a hypothetical question?
Merkle. Inc.
GB
Thanks for the quick reply.
Have a applications which is built on PEGA 8.1.
It has many dedicated queues(3 different queues, with 5 queue processes for each, we evenly distributed among 5), currently in production (there are about 300-500 ready to process queues, any given point of time on working days).
We did not find any issue as they get process, very quickly. (few seconds delay is fine for the application, near real time)
Now we are planning to upgrade to 8.3. After upgrade, will all the message of 3 different dedicated queues store in same Kofka Queue??
Will it give any performance impact?? or do we need to do some changes to create Kofka dedicated Queues ??
Above are the questions which flashed into our minds when we learned about new Queue Processer implementation in 8.3.
Thanks