Question
Morgan Stanley
IN
Last activity: 30 May 2018 5:18 EDT
Working of standard agent with less schedule interval and more queue items
Hi
I have a standard agent with
-> interval as 10 mins
-> Current queue items count is 20. Each item processing takes a minute, which means it takes 20 minutes to complete the entire processing
-> Max items to process is lets say 30
My question is.. when a agent starts and picks up items 1 by 1 and reaches 11 item it would have taken 10 mins..
1. Does it mean that the current agent would stop and hand over the processing to the new agent..
2. Will a new agent start or not? since there's already a agent present which is processing the queue items
3. Will the queue items be processed in sequence or in parallel
-- The machine is a single node
Thanks
Avinash
***Updated by moderator: Lochan to add Categories***
PEG
GB
This is actually quite a complex question, as it may need to factor in how many nodes you have and whether this agent is running on all nodes, how many batch requestors are running, and whether they are also trying to process other large jobs.
So let's pretend of the moment you have 1 application server node, this is the only agent running on that node, and you have the default number of batch requestors (this is 5 but can be amended through the <env name="agent/threadpoolsize" value="5" /> )
I'm also assuming that your agent doesn't "Max Records" value (as defined in the Agent rule)
When the agent next runs and collects the 20 records to process, this will then distribute those jobs to any free batch requestors. So if all 5 are currently idle then it will process the first 5 records/jobs in parallel, and as each batch requestor finishes the job it will then be allocated the next one of the 15 records/jobs remaining to be processed.
For the next run time; From observation this would initially calculated to 10 min from the current start time (if you examine this in SMA during one of these long runs). However, after it has completed all of the remaining jobs it will recalculate the time to 10 minutes after the current run finished.
This is actually quite a complex question, as it may need to factor in how many nodes you have and whether this agent is running on all nodes, how many batch requestors are running, and whether they are also trying to process other large jobs.
So let's pretend of the moment you have 1 application server node, this is the only agent running on that node, and you have the default number of batch requestors (this is 5 but can be amended through the <env name="agent/threadpoolsize" value="5" /> )
I'm also assuming that your agent doesn't "Max Records" value (as defined in the Agent rule)
When the agent next runs and collects the 20 records to process, this will then distribute those jobs to any free batch requestors. So if all 5 are currently idle then it will process the first 5 records/jobs in parallel, and as each batch requestor finishes the job it will then be allocated the next one of the 15 records/jobs remaining to be processed.
For the next run time; From observation this would initially calculated to 10 min from the current start time (if you examine this in SMA during one of these long runs). However, after it has completed all of the remaining jobs it will recalculate the time to 10 minutes after the current run finished.
But again, there are many assumptions in this small scenario which won't actually be true (as I doubt the node will only be running the one agent. And if you have the agent running on multiple nodes, I'd expect the jobs to be roughly distributed between those nodes).
Morgan Stanley
IN
Many thanks Laurence.
Assuming i have a single node.. When the agent starts for the first time.. it'll find 20 items in the queue and starts processing them..
While processing the items some more items lets say 5 items have joined the queue. Will the agent execution end after processing 20 or will it continuously process the items it keeps receiving..
Will the agent sleep only when there are no more items to process
PEG
GB
First, I would say that things can change across version.
My experience is the agent will only process the 20 items it found initially. For instance if the agent has to run for 3 minutes to process these 20 minutes and in the meantime you have 3 new items, these new items won't be processed on this run.
It is also possible the agent, which is supposed to run for instance every 60 seconds, takes far more than 60 second to process work items on its run. Then the agent carrying on running until it finished items it was supposed to processed to process and then next run start almost immediately. In fact, I have seen in the past an agent reported as hanging where in fact the agent was processing for hours a single work item (it was spending ages processing a huge email with a poor regex expression).
It is also possible for the agent to sleep even if there are items to process. There is a setting for the max number of item to process in a run. Let's say this is 1000. Now let's say there are 2000 queue items. The agent could wake up, process 1000 items and sleeps until the next run to process remaining items.