I found one client who had a Cloud environment where the issue occurred because of a faulty configuration base where a cleanup of older nodes did not happen and it picked random util nodes to attach and caused ES corruption. Web node accidently updates replica count which creates impacts Search Stability.
That issue was resolved in 8.6.6 under BUG-747182
Replica count updated only for index nodes
Every time a search node is restarted or reinitialized, a refresh command (refreshClients command) is sent to all nodes. A web node which is not a search node will start a reinitialization designed to reattach the node to ES cluster, and it was expected that the web node would not change information about the replica count since it is not responsible for storing data. However, the non-search node was changing the replica count due to a condition created when locking was not ready on the search node. This has been resolved with an update to insure replica count is updated only for index nodes in PegaSearchProviderImpl::reInitializeESNode.
Therefore it might be worthwhile to update to 8.6.6 to test your use case.
Posted: 8 months ago
Posted: 24 Mar 2023 6:08 EDT
Pawan Narain Singh (pawann)
We understand that major upgrades require a lot of planning and time. Patch updates, however, are considered an integral part of keeping Pega up-to-date. Pega Platform patches bundle the bug and security fixes over a given period into a cumulative release that improves the overall quality of the software without touching the environment infrastructure.