ReIndex after Archive process using InMemory Pipeline
We are currently scheduling pyPegaArchiverUsingPipeline job to Archive and Delete the workobjects from the app. Process indexes the resolved workobjects in secondary storage. But question is do we need to Reindex the Work- classes periodically to index for current population or the delete process will take care of this? Pega version is 24.1.3. Can someone shed some light on this? Thanks!
ai4process ltd
GB
Archival moves resolved work objects to secondary storage, and indexes them there and Deletion removes them from primary storage, but it doesn’t trigger reindexing of the remaining active cases.
You should periodically reindex the Work- classes to ensure that the current population (i.e., active cases) is properly reflected in search and reporting.
CSC
US
Thanks @RaviChandra for your response.Yes, we won't be running pyPegaExpunger for some time. So wanted to make sure the Elastic storage is optimally used. As per your response, then regular Re-indexing is needed to keep the SRS up to date with the current active population.
HCA Healthcare
US
you don’t need a periodic full reindex just because you archive and delete. When pyPegaArchiverUsingPipeline removes resolved cases, it also enqueues search-delete events so those work objects are dropped from Elasticsearch automatically. As long as your search cluster is healthy and the incremental indexer queue processors are running and clear, the index will reflect the current population without a scheduled rebuild. What this really means is your steady state should rely on incremental indexing, not recurring full reindex jobs. Reindex only when you see symptoms: “ghost” results, big gaps between database counts and search results, or failed items in the search QPs. After a large one-time archive burst, it’s normal for the delete backlog to take time to drain; give the indexers time before you assume it’s stale. Monitor Search landing page for index health and counts. Check the indexer queues for stuck messages and fix any broken items. If the backlog won’t clear or counts stay off, run a targeted rebuild for the affected Work- classes rather than the whole system. Use “Rebuild” or “Resave-and-reindex” for specific classes if only some case types are off. Avoid full system reindex unless you upgraded mappings, changed analyzers, or the index is corrupted. Make sure “Enable search indexing” is on for the classes you care about. Verify that delete events are not being filtered by any custom data transform in your archive pipeline. Confirm archive retains only metadata you need in secondary storage; don’t index archived payloads unless you truly need them. Keep your archive cadence predictable so indexer load remains smooth. Bottom line: the delete process keeps the index clean; you reindex only to correct drift, not as a routine after archive.
CSC
US
Thanks @Sairohith for sharing the info. That's very good to know. Yeah we will monitor the Search Index queue and Elastic storage as well after some initial runs(could be large as you said) for Archival job.