Declare Index issue after Pega upgrade
Yesterday we had Pega Upgrade in production and found couple of observation/exceptions which we didn't face in stage environment .
when upgrade is done ,after couple of hours we see lot of entries in index tables (Rule-Declare-Index) and all are duplicated (unfortunately no primary key on these index tables) also it happened for all resolved cases.
Not sure which event/action triggered this reindexing part as part of upgrade
Sharing few logs below but attached the complete log file and look at timestamps between 2023-06-27 15:08' AND '2023-06-27 16:49')
Attachment table:-
Yesterday we had Pega Upgrade in production and found couple of observation/exceptions which we didn't face in stage environment .
when upgrade is done ,after couple of hours we see lot of entries in index tables (Rule-Declare-Index) and all are duplicated (unfortunately no primary key on these index tables) also it happened for all resolved cases.
Not sure which event/action triggered this reindexing part as part of upgrade
Sharing few logs below but attached the complete log file and look at timestamps between 2023-06-27 15:08' AND '2023-06-27 16:49')
Attachment table:-
2023-06-27 15:09:13,633 [cols-AsyncExecutor-3] [ STANDARD] [ ] [ ] (writers.ScalarPropertiesWriter) INFO - Saving with updated declarative index information: DATA-WORKATTACH-FILE MyOrg-MyAPP-WORK L-94364!20191126T124825.221 GMT 2023-06-27 15:09:13,638 [cols-AsyncExecutor-3] [ STANDARD] [ ] [ ] (ols.readwriters.MaxIndexWriter) INFO - Max Index [-99999] < 0 # Resetted Top Index [0] # DATA-WORKATTACH-FILE MyOrg-MyAPP-WORK L-94364!20191126T124825.307 GMT has indexes exists but has no 'pzIndexes' property. 2023-06-27 15:09:13,666 [cols-AsyncExecutor-3] [ STANDARD] [ ] [ ] (writers.ScalarPropertiesWriter) INFO - Saving with updated declarative index information: DATA-WORKATTACH-FILE MyOrg-MyAPP-WORK L-94364!20191126T124825.307 GMT
Work Table
2023-06-27 15:09:39,482 [cols-AsyncExecutor-1] [ STANDARD] [ ] [ ] (writers.ScalarPropertiesWriter) INFO - Saving with updated declarative index information: MyOrg-MyAPP-WORK L-107906 2023-06-27 15:09:39,485 [cols-AsyncExecutor-1] [ STANDARD] [ ] [ ] (ols.readwriters.MaxIndexWriter) INFO - Top Index [5] > Max Index [4] MyOrg-MyAPP-WORK L-107907 2023-06-27 15:09:39,487 [cols-AsyncExecutor-0] [ STANDARD] [ ] [ ] ( internal.mgmt.Executable) ERROR - Only authenticated client may start this activity: RULE-OBJ-ACTIVITY RULE-OBJ-REPORT-DEFINITION PXRETRIEVEREPORTDATA #20220919T124254.899 GMT ReqID=BUBNXJNP1KBIU4RT3UNV8AI4ZQ0II374FA com.pega.pegarules.pub.PRRuntimeException: Error: You lack access required to execute RULE-OBJ-ACTIVITY RULE-OBJ-REPORT-DEFINITION PXRETRIEVEREPORTDATA #20220919T124254.899 GMT. at com.pega.pegarules.session.internal.mgmt.Executable.updateStepStatusAndLog(Executable.java:1086) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.mgmt.Executable.updateStepStatusAndLog(Executable.java:1076) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.mgmt.Executable.activityProlog(Executable.java:923) ~[prprivate-session.jar:?] at com.pegarules.generated.activity.ra_action_pxretrievereportdata_4487983b09d377f8f24eebaf846a174f.perform(ra_action_pxretrievereportdata_4487983b09d377f8f24eebaf846a174f.java:59) ~[?:?] at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:2872) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivity(Executable.java:11502) ~[prprivate-session.jar:?] at com.pegarules.generated.activity.ra_action_pxcallretrievereportdata_f81870511486f400512ef7cfa98f9945.step5_circum0(ra_action_pxcallretrievereportdata_f81870511486f400512ef7cfa98f9945.java:492) ~[?:?] at com.pegarules.generated.activity.ra_action_pxcallretrievereportdata_f81870511486f400512ef7cfa98f9945.perform(ra_action_pxcallretrievereportdata_f81870511486f400512ef7cfa98f9945.java:147) ~[?:?] at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:2872) ~[prprivate-session.jar:?] at com.pega.pegarules.exec.internal.declare.infengine.pages.DeclarativePageDirectoryImpl.runRule(DeclarativePageDirectoryImpl.java:1780) ~[prprivate-exec.jar:?] at com.pega.pegarules.exec.internal.declare.infengine.pages.DeclarativePageDirectoryImpl.runLoadActivity(DeclarativePageDirectoryImpl.java:508) ~[prprivate-exec.jar:?] at
Areteans
AU
If there is a failure in PRODUCTION, I would recommend raising Pega SR immediately, As your client purchases a Pega license, the Pega team will jump right in to help you resolve the issue.
But checking the class from the logs, it does not seem like live production. I would recommend revalidating and save the class and indexes, checking errors, and resolving them one by one.
If your index class does not have pzIndexs, then check the DB table and update the index class accordingly.
Skandiabanken
SE
@SiddhantJivane Thanks for the response ..Now i'm more looking at which job is causing the re-index part ?.
Do we really need to do revalidate and save for all class and indexes even though we restarted the servers after upgrade ?.
Areteans
AU
@BRAHMESH@ ,
Yeah, just to make sure if upgrade didn't cause any issue.
Hope you first upgraded the DB and then deployed Latest version of prweb.war.
Skandiabanken
SE
@SiddhantJivane Upgrade activity went well and we didn't face any issue with upgrade ..after upgrade is done , some indexing job was run which we don't know and updates all indexing records of cases ,attachments. usually recreateIndex activity should delete old records and create new one but here it's added new one even for resolved cases.