Question
Navistar INC
US
Last activity: 29 Apr 2020 12:01 EDT
Deployed rules not picking up post deployment.
Hello,
When the code is migrated to higher environments (TEST, QA and PROD) from DEV, the deployed rules are being picked up during execution.
All our environments have multiple node and only the node where the migration took place has the changes reflected but not the others.
I did find Support articles proposing to add DSS: Owning Ruleset: Pega-Engine; Setting Purpose: prconfig/systempulse/scanoffsetms/default ; Value: 60000. But this did not make any difference. Also when the nodes are being started, we did not find the entry is log file stating: (ccess.DatabaseImpl.SystemPulse) INFO - Using a pulse offset of 60000 ms.
So, is this DSS supposed to be working on Pega 7.4? If not, what would be our next steps resolving this discrepancy because restarting the environment every time post code migration is both resource and time consuming?
Reference Support articles:
https://community.pega.com/support/support-articles/rulenotfound-error-after-migration
Environment configuration: AIX 7.1, Websphere Proprietary information hidden, SQL Server 2014 running Pega 7.4 with Warranty and Manufacturing framework.
Thanks!
***Edited by Moderator Marissa to add SA IDs to post***
Accepted Solution
Centene Corporation
US
I just got an update on this issue.
The issue got resolved when the installation node is declared as Cluster rather Standalone and also set the DSS for SystemPulse. Hazelcast will sync all the nodes only if the installation node is declared as Cluster and ignores if standalone. It worked!!
Pegasystems Inc.
IN
Can you please see if System pulse agent is running successfully?
Also, check to see if there is no error on hazelcast end, ie, all related nodes are part of hazelcast internal cluster.
Navistar INC
US
System pulse is running on all the nodes and no issues with the Hazelcast too. I see all the nodes being part of the same cluster.
Pegasystems Inc.
US
Since the changes are picked up on the node where the import was done but not the other nodes, it sounds like it is related to the pulse process. Pega 7.4 uses Hazelcast by default. Can you check the cluster logs to see if all of the nodes show in the cluster? If there is an issue with communication between the nodes, then the changes will not get propagated to the other nodes.
Pegasystems Inc.
US
You could try setting the SystemPulse logging to ALL on each node. Then do an import and you should see a message such as "Broadcasting 1 System Pulse message" and it will show the details of the rule. Then on the other node, you should see a message about it being received (in a working scenario).
Pegasystems Inc.
US
1. Since the issue doesn't occur on the node performing the import, then most likely, it is System Pulse related.
2. Does restarting a node resolve the problems on that node?
If a restart of a node resolves the issue, then we can focus on how the rules are being invalidated and reassembled upon import, not on the saved rule itself.
3. Did you see any errors?
What kind of rules you have imported?
Is it only Stream rules that are exhibiting this behavior, or are there others?'
If it's just streams, then we can focus on stream assembly and/or the ABA cache after import.
Navistar INC
US
Hi Susan - Please find my responses with respect to your questions:
1) Yes, the node performing the import doesn't have any issue.
2) Yes, restarting the environment resolves the problem and all nodes have the new imported changes reflected.
3) I don't see any errors in either of PegaRULES or Alert log.
Issue is not specific to Stream rules, we found old Report Definitions and activities getting picked up during execution.
Coforge DPA
GB
Hi
Can you check if your database is marked as "Persistent". You may take help of your DBA to find that out.
Ideally it should be persistent. We faced a similiar issue when we upgraded to 7.4.
Regards
Bhavya
Navistar INC
US
Bhavya - Sorry I did not quite understand when you say database being marked as 'Persistent'.
We have no issues saving the rules to the DB during import, the issue is with the older rules picking up during runtime as referenced from cache.
Thanks!
Centene Corporation
US
Is this Issue resolved? We are facing the similar kind of issue after we upgraded to 8.1.
we tried the above mentioned DSS to set systempulse and it did not work.
Accepted Solution
Centene Corporation
US
I just got an update on this issue.
The issue got resolved when the installation node is declared as Cluster rather Standalone and also set the DSS for SystemPulse. Hazelcast will sync all the nodes only if the installation node is declared as Cluster and ignores if standalone. It worked!!
Centene Corporation
US
I see the issue came back again after 8.2.2 patch upgrade.
Manual Deployment - Works
Jenkins Automated deployment - HTML rules needs revalidate and save.
I figured out a new DSS (Pega-Engine - initialization/pulse/transport="cluster")
from the Pega forums which is working fine but still i think it's not reliable as HTML rules are not picking up or required revalidate and save after deployment into higher environments.
I am planning to go with an SR and will post any further updates regarding this issue. Meanwhile if anyone facing this issue please suggest any fix that worked.
Pegasystems Inc.
US
US
@PhaniPraturi can you confirm if your completely resolved by adding DSS initialization/pulse/transport="cluster" Setting ? Or if you had to perform any other changes from the SR investigation?
We are seeing a similar issue in 7.4 with Decision table updates not picking up on few nodes, it is very rare but impacts our business.
Thanks in advance for your respsonse.