Error: Queue Processor: <QP name> can not be started. Nodes of Web User type are not present in cluster
Hi,
Greetings. Hope you are well! I started getting error when trying to start a Queue Processor as below, I am not sure if there is a limit on how many Queue Processors can be associated with a Node Type. Can someone please let me know what is causing this error and how to resolve this?
We have 16 Web User Nodes in the cluster and all these QPs have 20 partitions altered via pxAlterStreamPartitions.
Regards,
Bharat
Accepted Solution
Accenture
GB
Updated: 8 Nov 2023 18:58 EST
Areteans
AU
It's not ideal to run the queue processors on web user. From Pega 8.7 onwards it is recommended to have 6 partitions, More partitions can create issues. Refer below article.
https://docs-previous.pega.com/system-administration/87/queue-processor-faq
If possible,
A. can you try setting the max partition to 6,
B. Perform full restart, delete kafka-data folder content from app server/host. [This folder contains the partitions, as you currently you have 20 partitions there will be 20 folders with the same QP name which represents partition- so we will need to delete it so that Pega can then create 6 new partitions]
C. Truncate below table:
truncate table pegadata.pr_data_stream_node_updates;
truncate table pegadata.pr_data_stream_nodes;
truncate table pegadata.pr_data_stream_sessions;
Hope this helps!
Accenture
GB
@SiddhantJivane Sure, but there is no mention that QPs can only be associated with Background Processing Nodes, also the FAQs mention that though the default partitions are 6, it can be increased if there is a heavy throughput.
Regards,
Bharat
Proximus
BE
Hello,
QPs need to be associated to a NodeType, it can be anything but WebUser sounds like nodes for user so best practises tell to avoid background processing on nodes where you've got users. But technically, there's no restriction.
Here, the error tells that there's no WebUser nodes, it doesn't mention any limitation.
I guess you checked that there's indeed WebUser nodes and that you well checking for the correct cluster (admin studio should give you all needed info)
If possible, I would suggest to change QP node type to any of the default (Stream, Search, BackgroundProcessing) and then check whether it starts or not.
If you want to see whether it's a limitation with the number of QP, then you could try stopping several (all) of existing QP and then try to start this failing one. if OK then it's limitation, if standalone it fails then it's not linked to a limitation.
You could also add node type Custom1 to one of your server and assign this QP to Custom1. You can even keep this Custom1 to make validation of any new QP/JS when needed.
Still, I would first double check your cluster status and make sure that all required access/defintion are well defined for this specific QP.
You've got 16 web user nodes but are they well defined with nodetype WebUser node type...? there's no constraint in having user connected and WebUser nodetype.
Hello,
QPs need to be associated to a NodeType, it can be anything but WebUser sounds like nodes for user so best practises tell to avoid background processing on nodes where you've got users. But technically, there's no restriction.
Here, the error tells that there's no WebUser nodes, it doesn't mention any limitation.
I guess you checked that there's indeed WebUser nodes and that you well checking for the correct cluster (admin studio should give you all needed info)
If possible, I would suggest to change QP node type to any of the default (Stream, Search, BackgroundProcessing) and then check whether it starts or not.
If you want to see whether it's a limitation with the number of QP, then you could try stopping several (all) of existing QP and then try to start this failing one. if OK then it's limitation, if standalone it fails then it's not linked to a limitation.
You could also add node type Custom1 to one of your server and assign this QP to Custom1. You can even keep this Custom1 to make validation of any new QP/JS when needed.
Still, I would first double check your cluster status and make sure that all required access/defintion are well defined for this specific QP.
You've got 16 web user nodes but are they well defined with nodetype WebUser node type...? there's no constraint in having user connected and WebUser nodetype.
Well, with above, you've got few options to debug if the logs are not showing more detailed info.
Regards
Anthony
Accepted Solution
Accenture
GB
Hi, Greetings. As i have mentioned there is no restriction on what Node Type a QP can be associated with but as per best practices, it not recommended to associate it with a Web User. However the error i have mentioned has nothing to do with node type association or availablity of nodes in a cluster (these are in cluster). There is a DSS setting to have maximum of 10 default Data Flow runs on Web User and Batch however unlimited on BackgroundProcessing, Search and RTDG. As QP reply on Data flows in the backend, this default setting was causing the error.
Resolution: Increase the default data flow runs active at a time to >10 (see below)
Any questions, please do not hesitate to message me.
Regards,
Bharat
Hi, Greetings. As i have mentioned there is no restriction on what Node Type a QP can be associated with but as per best practices, it not recommended to associate it with a Web User. However the error i have mentioned has nothing to do with node type association or availablity of nodes in a cluster (these are in cluster). There is a DSS setting to have maximum of 10 default Data Flow runs on Web User and Batch however unlimited on BackgroundProcessing, Search and RTDG. As QP reply on Data flows in the backend, this default setting was causing the error.
Resolution: Increase the default data flow runs active at a time to >10 (see below)
Any questions, please do not hesitate to message me.
Regards,
Bharat