Question
Royal Bank of Canada
CA
Last activity: 1 Nov 2021 16:53 EDT
Can we see the queued items into a queue processor?
Hi All,
I have a question regarding the queue processor. As these queue processors are new in Pega we have no idea what issues we may get once in production. One question I was asked when I suggested queue processor for a batch processing job is what happens to the data queued into the queue processor in case of a system node went down. I think Pega will automatically process the requests in Kafka topic once the node is up and running but can we see the queued items if we want to check what items exist in queue? Not from the admin portal but from back ground like in db table or some where?
Maantic
IN
There are 2 classes and respective tables you can check.
System-Message-QueueProcessor-DelayedItem
System-Message-QueueProcessor-BrokenItem
Check class mappings to get the table names.
Updated: 25 Oct 2021 12:06 EDT
Royal Bank of Canada
CA
@Avik Mitra Hi Avik, Thank you for your prompt reply. I am looking into the scenario where the node is down and to see what happened to the items queued. Let's say we have 100 items in the queue and the node went down in between without finishing the processing of all the items in the queue, then where can we see the reaming items unprocessed? Could you please let me know if you have any idea?
Thanks,
Durga Prasad
Maantic
IN
If stream service is not available, entries are made into delayed item table. Try stopping the stream node from infrustructure services and queue an item using a queue processor. You will see an entry is made into delayed queue.
Once you start the stream node, you will notice that entries are gone from the table.
Royal Bank of Canada
CA
@Avik Mitra Thank you very much for your reply. I am very sorry my delay in response. Ya I will try that and see how it works.
Aaseya IT services Pvt Ltd
SA
When Stream node is down , messages are queued to database as an instance of System-Message-QueueProcessor-DelayedItem , Once node is up and available , the entry from the databased is dequeued to the topic or kafka folder for queue processor to process.
Broken entries will be available as an instance of System-Message-QueueProcessor-BrokenItem
You can see the broken entries and ready to process and other entries in the admin studio . I have attached the same here.
Royal Bank of Canada
CA
@ArshadPashaMohammed Hi Arshad, Thank you for your prompt reply. In case of a node down the items will be moved to delayed queue and will be processed once the node is up right? And this is also same for exiting items in the queue? For example when we have 100 items in the queue already and the node went down in the middle of the process where can we see the remaining items in the queue? Could you please if have any idea please let me know?
Thanks,
Durga Prasad
Aaseya IT services Pvt Ltd
SA
Till node is up , the items which are getting processed you can check from the admin studio as shared as an attachment in earlier conversation.
As soon as node goes down , the entries will be put to table instead of kafka folder and you can check those entries in the table pr_sys_delayed_queue.
let me know if any other details are required.
Royal Bank of Canada
CA
@ArshadPashaMohammedI am very sorry for the delay in my response. Thank you very much for your reply which helps me a lot.