Upgrade Testing Validation
What is the purpose of this validation?
During the basic upgrade process, the client have to test all the applications for functional correctness within either 15 days for a minor upgrade or 30 days for a major upgrade. This testing will be conducted on the cloned and upgraded Staging environment. When the client notifys Pega that they have completed their testing and is ready to upgrade Production, a validation check will be invoked by the Pega as part of the upgrade process to determine if the client has done sufficient testing. If the client have met the criteria for testing completeness of their application, the Production upgrade process will proceed and will pause otherwise giving the client a chance to complete their application testing.
What gets checked as part of this validation?
This validation checks for the following
- Case type validation
- Verifies if at least one case instance of every case type exists across all the client applications in the system – This is to ensure that all case types have been tested at-least once.
- Application login validation
- Verifies if there is at-least one user login exists for any one client application. This is a minimal check to make sure that user level testing has been conducted.
The data for both these checks is taken from the time the environment is provisioned till the current date, when the Production upgrade is scheduled, or for last 60 days, whichever is earlier.
All Pega platform and strategic applications are excluded from this validation (applications as well as case types belong to them). Only those applications that are created by the client are considered for this validation.
How can this validation fail?
This validation fails when either or both the validation checks fail. Following table can be referred to understand it better.
What is the overall work flow of this validation?
This validation is integrated to the Pega platform’s upgrade process and runs automatically just prior to the Basic upgrade of Production environment. The client responsibility is as follows
Client performs application computability testing and validation on the upgraded, cloned staging environment during the prescribed upgrade validation interval of time.
This forensic validation runs on the customer’s system every day and generates a report with the validation result up to the current day. The results of each day’s validation can be viewed by adding and switching to Pegaplatformvalidator access group and visiting the following landing page in the system.
Configure -> System -> Settings -> ForensicsValidation
The 3 tiles on the top of the page shows the result of the latest validation.
Validation status tile shows the overall validation result (PASSED/FAILED)
Case type tile shows the validation result of case type check (PASSED/FAILED)
Application login tile shows the validation result of application login check (PASSED/FAILED)
By clicking on the top most report under “Validation history”, details of the case types not tested and applications not logged in can be seen in a separate tabs, as shown in the following images.
Above page shows the details of case type check in the validation. Clicking on the “Applications login” tab shows the details related to the application login check as shown in the following image.
It is advised that client keeps checking this page on a regular basis to see what is pending to be tested.
When the client indicates that they have finished their testing and ready to upgrade, Pega’s upgrade process will invoke this forensic validation service to fetch the latest report to see if client has done enough testing.
What happens when the validation fails?
When forensic validation service returns the result as “FAILED”, the upgrade process stops from proceeding further with production upgrade.
The Service engineer responsible for upgrading a client’s production environment will check the error message and the recommendation returned by the validation service and establishes a dialog with the client indicating the corrective action by supplying a unique “Validation ID” of the report which is considered for the validation.
Client needs to add and switch to the Pegaplatformvalidator access group and visit the landing page “Configure -> System -> Settings -> ForensicValidation” to understand the failure details of the validation and ensure enough testing is done to make the validation passed. Validation ID supplied by service engineer would be handy to locate the exact report being referred, in case more testing done later by the client and more reports exist on the system.
What is the recommendation?
Ensure that the case types across all the client applications are tested properly and login to each of the applications with users from all applicable access groups on the cloned staging environment to prevent functional issues post upgrade.