@FerdinanP Very interesting question. I couldn't find anything on the number of topic restrictions in Kafka. So I assume you could create as many queue processors as you want. Plus, there is no way to stop an application from using the standard Queue processor. What number are you at? Are we talking thousands of Queue Processors?
Posted: 1 year ago
Posted: 17 May 2022 3:42 EDT
Aneesha Gundmi (AneeshaG)
I think there is no upper limit for Queue processor in a cluster. It just boils down to the fact that each cluster can support max of 20 partitions. Also, thread count is not equal to partitions. You can check prconfig/dsm/services/stream/pytopicpartitionscount/default value to check the partition count defined.