In Production we have zone 1, zone2.

Zone 1 - Active Schema --> Rule Schema 1 , Common Data Schema

Zone 2 - Inactive Schema --> Rule Schema 2 , Common Data Schema

Zone 1 - Active, which takes traffic all the time.

Zone 1, Zone 2 will be in synch all the time except for new feature deployment.

When we deploy new features to avoid impact to incoming traffic. we deploy the code to Zone 2(Inactive schema).

Incrementally/Manually we will redirect the traffic to Zone 2 and validate the latest code. Post Successful validation code will be synched to Zone 1

Prerequisite for zone 2 (Inactive schema Deployment) - If the new feature absolutely have only rules but not any data schema related changes.

If any changes to Data tables, DSS, Agents we will go with Zone 1 (Active Schema) deployment.

Recent Scenario - Latest Deployment - Validation is still in progress in zone 2. Meanwhile it has caused impact to incoming traffic at Zone 1 (Active Schema).

Root Cause :

Recent deployed Package has an existing "Service-Rest" rule updated with new headers. Pega recreated its corresponding Data-Admin-Service-Package instance.

Though the changes are related to Rule schema, underlying Pega Create/Update Data instance(Service Package) which is in common data schema.

Zone 1 traffic is picking Data-Admin-Service-Package rule from common data schema which in turn referring to Latest Service-Rest which is available only in Zone 2 (Inactive Schema) and throwing errors.

Solution for now :So we have updated our prerequisite check list to also include Service-* rules. And in future if any similar changes we will always go to Zone 1 i.e. Active zone deployment.

Question : We are now trying to find out all such similar rules for which Pega creates / updates Data Instances in background. Please advise..