Tune up Environment/Strategies after restart
Hello,
in our project, we have deployed NBA application which executes strategies to get best offers for Customers.
Now, we have observed that when nodes are working continuously (without breaks) for a few days, response time becomes poor. For instance, Prod nodes were restarted two days ago; after restarting nodes, average response time was 0.6 seconds, now average response time are more than 2 seconds. To measure response time, we take the time since first step of main activity (called from HTTP request by end users) is executed until last step is done (before http response is sent).
In order to keep good response time (less than 1 second), we are planning to restart Prod nodes every day at users-free window, Pega Cloud will be in charge of this action; but we have one issue and we need your help to perform daily restart:
After every restart, when main activity is called for the first time, response time is very high (around 90 seconds), and in case of Prod environment (4 nodes working behind Load Balancer), it happens every node receives the first request.
As we need to avoid that end users experience high response times of the first request, we have thought one approach to a solution (not started), but we have several doubts about their usefulness:
Hello,
in our project, we have deployed NBA application which executes strategies to get best offers for Customers.
Now, we have observed that when nodes are working continuously (without breaks) for a few days, response time becomes poor. For instance, Prod nodes were restarted two days ago; after restarting nodes, average response time was 0.6 seconds, now average response time are more than 2 seconds. To measure response time, we take the time since first step of main activity (called from HTTP request by end users) is executed until last step is done (before http response is sent).
In order to keep good response time (less than 1 second), we are planning to restart Prod nodes every day at users-free window, Pega Cloud will be in charge of this action; but we have one issue and we need your help to perform daily restart:
After every restart, when main activity is called for the first time, response time is very high (around 90 seconds), and in case of Prod environment (4 nodes working behind Load Balancer), it happens every node receives the first request.
As we need to avoid that end users experience high response times of the first request, we have thought one approach to a solution (not started), but we have several doubts about their usefulness:
- Create an activity to tune up environment by calling our main activity (Can we schedule activities after restart nodes? Will be this approach useful to tune up all nodes?)
Based on your experience, could please advice us to achieve our need?
Regards,
David
Pegasystems Inc.
US
Hi David,
You may want to take a look at the static assembler for avoiding slowness on the first run through your functionality:
https://community.pega.com/sites/default/files/help_v719/procomhelpmain.htm
https://pdn.pega.com/how-use-static-assembler-utility-pre-assemble-rules-application
Hope this helps!
-Kurt
Pegasystems Inc.
GB
Static Assembler is used to assemble and compile rules - as the name suggests.
When an active system is restarting rule assembly should not be required. Pega should load a percentage of the MRU rules on startup and thereby "initialise" the cache.
To understand this issue we should look at the logs in more details.
The symptoms as described by David could indicate a decline in the overall GC performance of the JVM. Again the GC data will need to be reviewed along with a more holistic review of the environments.