Scheduled Agents with Auto queue management
I have a scheduled Agent 'ProcessReportQueues', 5 entries of this that call activities ProcessQ1, ProcessQ2, ..Q5 from the class System-Queue-DefaultEntry.
All have Interval '30sec', are of mode 'Standard', Category is 'Uncategorized' and 'Auto-queue management' checked.
1. Are items within a queue processed first-come-first-serve?
2. Does each task have to complete before the next one is picked within the same queue?
3. The PegaRules-Batch-<number> that we see in the log, does this Batch# correspond to particular queue everytime?
For example, does Batch3 always refers to a particular queue, say 'ProcessQ2'
Or can Batch3 can sometimes be for a task in ProcessQ1 and sometimes for ProcessQ2?
4. I have a Declare Trigger on 'CommittedSave' that calls an activity which pushes the item to the queue for the agent 'ProcessReportQueues' agent.
When a work object say 'WO-ABC' is CommittedSave twice within few seconds, it's pushed into 2 queues, say ProcessQ3 and ProcessQ4.
However, ProcessQ3 and Q4 may finish the task in order different than the original call was made and is giving incorrect results.
For example, if WO-ABC has a property set to 'pOld' and CommittedSave, and then property set to 'pNew' and CommittedSave, both occuring within few seconds
I have a scheduled Agent 'ProcessReportQueues', 5 entries of this that call activities ProcessQ1, ProcessQ2, ..Q5 from the class System-Queue-DefaultEntry.
All have Interval '30sec', are of mode 'Standard', Category is 'Uncategorized' and 'Auto-queue management' checked.
1. Are items within a queue processed first-come-first-serve?
2. Does each task have to complete before the next one is picked within the same queue?
3. The PegaRules-Batch-<number> that we see in the log, does this Batch# correspond to particular queue everytime?
For example, does Batch3 always refers to a particular queue, say 'ProcessQ2'
Or can Batch3 can sometimes be for a task in ProcessQ1 and sometimes for ProcessQ2?
4. I have a Declare Trigger on 'CommittedSave' that calls an activity which pushes the item to the queue for the agent 'ProcessReportQueues' agent.
When a work object say 'WO-ABC' is CommittedSave twice within few seconds, it's pushed into 2 queues, say ProcessQ3 and ProcessQ4.
However, ProcessQ3 and Q4 may finish the task in order different than the original call was made and is giving incorrect results.
For example, if WO-ABC has a property set to 'pOld' and CommittedSave, and then property set to 'pNew' and CommittedSave, both occuring within few seconds
Now, queue ProcessQ3 would have picked pOld and wrote to a target database and ProcessQ4 would have picked pNew and wrote to the same target database.
But since ProcessQ3 and Q4 are independent we have cases where Q3 finished the task later than Q4 and hence pNew was written to the target database first and quickly overwritten with pOld.
This is making our target data incorrect.
How to resolve this issue?
How can we force item with same WO name to be pushed to the same queue if it already exists in a queue?
For example when queued to Agent 'ProcessReportQueues', can it check if the WO already exists (because of previous queuing) in any of the ProcessQ2...Q5 and if does, then push it to the same queue?
Thanks
Accepted Solution
Pegasystems Inc.
US
I assume you are using the multiple agents to process at a higher rate. It seems your requirement maybe could be met by checking a certain update/save time stamp to ensure the current update to the database isn't older than the other update with your agent. You could also split out the agents to not pull from the same queue and ensure each id is routed to certain queue based on same processing like id number modulo 5 goes to queue number modulo 5. I suspect even the later strategy you could encounter conditions where record based a lock conflict is processed out of order.
Pegasystems Inc.
US
Hello,
Just want to get some clarification on a couple of the items mentioned:
- How many nodes is this agent enabled on?
- Not sure I follow what you are saying here:
- "I have a scheduled Agent 'ProcessReportQueues', 5 entries of this that call activities ProcessQ1, ProcessQ2, ..Q5 from the class System-Queue-DefaultEntry." Does this mean you 5 instances of the agent running on a single node like ProcessReportQueues1 calls ProcessQ1 or a single agent that within the processing activity can call one of 5 other sub activities?
- Questions
1. Are items within a queue processed first-come-first-serve? Processed based on minimum date time for processing which can be effected by a few facets of object processing an lock availability, but for the most part yes.
2. Does each task have to complete before the next one is picked within the same queue? This is dependent on the questions I was seeking clarity about above. If another agent by node or duplicate on the same node with a number ofset is reviewing the same queue they can pickup another entry as soon as the first entry is reserved not completed.
Hello,
Just want to get some clarification on a couple of the items mentioned:
- How many nodes is this agent enabled on?
- Not sure I follow what you are saying here:
- "I have a scheduled Agent 'ProcessReportQueues', 5 entries of this that call activities ProcessQ1, ProcessQ2, ..Q5 from the class System-Queue-DefaultEntry." Does this mean you 5 instances of the agent running on a single node like ProcessReportQueues1 calls ProcessQ1 or a single agent that within the processing activity can call one of 5 other sub activities?
- Questions
1. Are items within a queue processed first-come-first-serve? Processed based on minimum date time for processing which can be effected by a few facets of object processing an lock availability, but for the most part yes.
2. Does each task have to complete before the next one is picked within the same queue? This is dependent on the questions I was seeking clarity about above. If another agent by node or duplicate on the same node with a number ofset is reviewing the same queue they can pickup another entry as soon as the first entry is reserved not completed.
3. The PegaRules-Batch-<number> that we see in the log, does this Batch# correspond to particular queue everytime?
For example, does Batch3 always refers to a particular queue, say 'ProcessQ2'
Or can Batch3 can sometimes be for a task in ProcessQ1 and sometimes for ProcessQ2? The idenifier is only static for a given processing cycle once it goes to sleep no garantees of the same tag next wake up.
4. I have a Declare Trigger on 'CommittedSave' that calls an activity which pushes the item to the queue for the agent 'ProcessReportQueues' agent.
In regards to this question the clarity on the above questions is necessary to determine alternatives.
Pegasystems Inc.
US
Hi Stilr, I uploaded screenshot of the agent schedule now.
PRPC is on a 3 node system. The agents however run only on one node.
As you can see from the screenshot, we have 5 Agents with the same name 'ProcessHMReportQueues' which call different activities (each of these activities ultimately call a common activity though).
Let's say a work object WO-ABC was CommittedSave twice within few seconds. Let says at time T1 and T2, T1<T2. The declare trigger calls an activity which will push 'WO-ABC' for Agent 'ProcessHMReportQueues'
Now since we have multiple Agents with the same name 'ProcessHMReportQueues', lets say one (T1) was pushed to highlighted row 4 (with activity ProcessHMreqportQueues1) and the other (T2) row 5 (with activity ProcessHMReportQueues3).
If queues from row 4 and 5 run in parallel, then there's a good chance that row 5 (T2) finished processing earlier than row 4 (T1) and hence only the change at T1 is effective.
We are pushing work objects changes to target reporting database. T1 should have been written first to target database and then overwritten with T2. However, because of the parallel queues T2 could get written first and then overwritten by T1 (which makes the reporting db inaccurate)
Accepted Solution
Pegasystems Inc.
US
I assume you are using the multiple agents to process at a higher rate. It seems your requirement maybe could be met by checking a certain update/save time stamp to ensure the current update to the database isn't older than the other update with your agent. You could also split out the agents to not pull from the same queue and ensure each id is routed to certain queue based on same processing like id number modulo 5 goes to queue number modulo 5. I suspect even the later strategy you could encounter conditions where record based a lock conflict is processed out of order.
Pegasystems Inc.
US
Option 1. Today we don't read the db record, we just write the copy of the source to target (db).
If we have the same work object name in 2 queues, and both are processing parallely and none have saved to the target db yet, then both will read the same old db record (or none if it doesn't exist) and then both will go for write to db and can write in the wrong order. The only way to guarantee this doesn't happen is that one queue shouldn't read the target db reocord until the other has committed first, and I am not sure how to guarantee this.
Also, today we don't read the target db record, we simply write to it. Reading db record (to get timestamp) would be an overhead.
Options 2.
I already thought about this. I can have the AgentNames different, say ProcessHMReportQueues0,1,...5.
Depending on the work object numeric suffix modulo 5, it can be pushed to the corresponding named agent with suffix 0-4 (Queue 5 can be used of non-numeric suffix or special cases). How can we encounter lock conflict here? 2 items for the same work object name will go to the same queue and a queue will process items sequentially, so it shouldn't process the 2nd instance until the first one is committed, right?
Option 1. Today we don't read the db record, we just write the copy of the source to target (db).
If we have the same work object name in 2 queues, and both are processing parallely and none have saved to the target db yet, then both will read the same old db record (or none if it doesn't exist) and then both will go for write to db and can write in the wrong order. The only way to guarantee this doesn't happen is that one queue shouldn't read the target db reocord until the other has committed first, and I am not sure how to guarantee this.
Also, today we don't read the target db record, we simply write to it. Reading db record (to get timestamp) would be an overhead.
Options 2.
I already thought about this. I can have the AgentNames different, say ProcessHMReportQueues0,1,...5.
Depending on the work object numeric suffix modulo 5, it can be pushed to the corresponding named agent with suffix 0-4 (Queue 5 can be used of non-numeric suffix or special cases). How can we encounter lock conflict here? 2 items for the same work object name will go to the same queue and a queue will process items sequentially, so it shouldn't process the 2nd instance until the first one is committed, right?
Option 3.
From the screenshot, rows 3-7 have the same AgentName, but they all have their own queues, right? How can I get the actual queue name used internally?
Keeping the agent names as it is, can I lookup in which internal queue a WO already exists (if) and then push the new request to the same queue?
For example, out of the 5 same AgentNames, how can I check which one already has WO-ABC (if it already does) and how can I push new WO-ABC request to the same queue (out of 5 possible from the same AgentName)?
Thanks,
-Sri