Question
Accenture
AE
Last activity: 15 Jan 2024 8:43 EST
Pega 8.8 Migration - Cassandra Idle Load issue
Dears
We have recently migrated from 7.3 to Pega 8.8 and are currentl in the testing phase.
One of the issues which has been raised by the client is the Cassandra load issue even when the system is NOT LIVE. In the error logs we observe that there is only 1 query which is repeatedly causing the load on Cassandra. Following is the query:
'WARN [ReadStage-2] 2024-01-08 17:15:20,335 ReadCommand.java:592 - Read 1071 live rows and 4542 tombstone cells for query SELECT d FROM adm_commitlog.adm_meta WHERE pk = 'newkeys' LIMIT 5000 ALLOW FILTERING; token -7088964478881828935 (see tombstone_warn_threshold)'
It seems that the load is coming due to queries being generated by ADMS. The concern is to understand that why is there a constant load on Cassandra even when we havnt gone LIVE with all the ADM models and campaigns ?
I am attaching the screen shots from APP dynamics containing the load monitoring for your reference.
Requesting assistance on the same
Thanks and Regards
Vaibhav Chandra
Pegasystems Inc.
NL
Hi @VaibhavC3208,
This query is performed by the ADM server node as part of the process to find out which models require an update, irrespective of the number of models that are currently active. If you have a live ADM server node, then this query will be performed on a regular basis.
That being said, the adm_commitlog.adm_meta table is very small, and usually does not extend beyond a few thousand records and total size of few dozen MBs. As you can see in the logs, the number of queries performed are also limited. Though they happen regularly, it should only happen at a low frequency, in this case only 4-5 queries per minute.
Given this we do not expect this to cause any considerable load on the Cassandra cluster.
As the metrics are reporting ~600 operations per second, other queries than the one you report have to be responsible for this. It could help to look at metrics on a per keyspace, or per table, basis.
Best, Tom