Question
Volksbank
PL
Last activity: 11 Sep 2024 8:06 EDT
pzCheckProfileMergeKafkaAlert and pzMergeMetricsConfigPulse Job schedulers
Hello Pega,
After updating the software to the Infinity version, we began to notice some impact on performance, caused by those Jobs.
I could not find any articles mentioning them, but quick analysis tells me they might be some leftovers after the migration itself:
-
Rules89To23 - this is the name of the operator who has created them
-
com.pega.decision.profile.merge.apibridge.metrics.MergeMetricsHelper.getInstance().handlePulse() - this is the funcion called by it
Some time ago, those Jobs were disabled on lower environments, can you please provide an explanation of what are they for and if are they needed at all?
With kind regards,
Marek
Accepted Solution
Pegasystems Inc.
GB
@MarekG16680293 INC-B20196 was resolved with the following analysis:
--------------------------------------------
Both Job Schedulers are related to Profile Merge feature in Platform, here’s a description of this feature for your reference:
Data flow
The merge processor is a data flow created to process the merge requests. The merge processor reads records from the merge stream, loads profile definition determined by the merge request parameters, and invokes merge processing on the data sources associated with the profile:
The source shape is defined as the merge records stream – pxMergeStream data set in the Data-Decision-Profile-MergeRequest class.
The destination is the MergeStageProcessor shape.
Because of the need to aggregate records from multiple applications, the data flow runs in the System Runtime Context (SRC) but, when in the context of processing a specific merge request, it can run under a specific access group because multiple applications can include the same data source.
Profile cache
@MarekG16680293 INC-B20196 was resolved with the following analysis:
--------------------------------------------
Both Job Schedulers are related to Profile Merge feature in Platform, here’s a description of this feature for your reference:
Data flow
The merge processor is a data flow created to process the merge requests. The merge processor reads records from the merge stream, loads profile definition determined by the merge request parameters, and invokes merge processing on the data sources associated with the profile:
The source shape is defined as the merge records stream – pxMergeStream data set in the Data-Decision-Profile-MergeRequest class.
The destination is the MergeStageProcessor shape.
Because of the need to aggregate records from multiple applications, the data flow runs in the System Runtime Context (SRC) but, when in the context of processing a specific merge request, it can run under a specific access group because multiple applications can include the same data source.
Profile cache
The merge processor caches an instance of the profile so that it does not need to retrieve it from the database for every merge request – the profile provider cache expiration time DSS defines the profile caching window (default is 5 minutes). If any changes have been done to the profile in CPD, they are picked up the next time the merge processor refreshes the cache, and there is no need to restart the merge processor run to apply changes to the profile. As each data source is merged, the merge log is updated to reflect it in order for us to be able to determine what has and what has not been merge.
How does the merge processor state watcher work?
The merge process is a managed data flow run that is not resumed automatically by the data flow framework in case of failure. This is the technical reason behind the MergeProcessorStateWatcher implemented in Java code to periodically checks the state of the merge processor data flow run and, if the state is FAILED, resume the run.
The MergeProcessorStateWatcher check method is invoked by the Platform function pzMergeProcessorWatcher(), which is contained in the pzRunProfileMergeKafkaAlertCheck activity executed periodically by the pzCheckProfileMergeKafkaAlert job scheduler. By default, it runs every 3 minutes, and this can be configured by the merge processor state watched DSS (profile/merge/mergeProcessorWatcher). The Job Scheduler pzMergeMetricsConfigPulse also works in the same context for the same feature for reporting merge processor metrics.
We have created an internal BUG item (BUG-853551) to have the external documentation updated for these Job Schedulers – this is a work in progress and should be completed soon.
-------------------------------------
Pegasystems Inc.
GB
@MarekG16680293 The jobs you mentioned seem to be related to the Decisioning capabilities of Pega Infinity. They might be involved in handling metrics for decisioning profiles. However, without more specific information about these jobs and their impact on your system, it's hard to provide a definitive answer. It's recommended to consult with Pega Support or a Pega Certified Senior System Architect for a detailed analysis of your system's performance and the role of these jobs.
⚠ This is a GenAI-powered tool. All generated answers require validation against the provided references.
Pega Platform 8.5.1 Patch Resolved Issues
Pega Platform changes with update impact > Known update impact in Pega Pla
Pega Platform 8.8.3 Patch Resolved Issues
Please log a support issue via the MSP and provide the INC here
Pegasystems Inc.
GB
@MarekG16680293 I see that INC-B20196 was logged for this:
Our support team recommended the below:
------------------------------------
The followig OOTB job schedulers pzCheckProfileMergeKafkaAlert and pzMergeMetricsConfigPulse are added newly as part of Pega Infinity/Pega 8.8.x version and they are part of Pega-DescisionEngine ruleset,
If you are not using any decision related functions you can disable them from Admin studio.
Currently I can see from PDC that these Job schedulers are disabled.
Please let me know if the issue is not coming up anymore so that I can go ahead and close this INC.
------------------------
Accepted Solution
Pegasystems Inc.
GB
@MarekG16680293 INC-B20196 was resolved with the following analysis:
--------------------------------------------
Both Job Schedulers are related to Profile Merge feature in Platform, here’s a description of this feature for your reference:
Data flow
The merge processor is a data flow created to process the merge requests. The merge processor reads records from the merge stream, loads profile definition determined by the merge request parameters, and invokes merge processing on the data sources associated with the profile:
The source shape is defined as the merge records stream – pxMergeStream data set in the Data-Decision-Profile-MergeRequest class.
The destination is the MergeStageProcessor shape.
Because of the need to aggregate records from multiple applications, the data flow runs in the System Runtime Context (SRC) but, when in the context of processing a specific merge request, it can run under a specific access group because multiple applications can include the same data source.
Profile cache
@MarekG16680293 INC-B20196 was resolved with the following analysis:
--------------------------------------------
Both Job Schedulers are related to Profile Merge feature in Platform, here’s a description of this feature for your reference:
Data flow
The merge processor is a data flow created to process the merge requests. The merge processor reads records from the merge stream, loads profile definition determined by the merge request parameters, and invokes merge processing on the data sources associated with the profile:
The source shape is defined as the merge records stream – pxMergeStream data set in the Data-Decision-Profile-MergeRequest class.
The destination is the MergeStageProcessor shape.
Because of the need to aggregate records from multiple applications, the data flow runs in the System Runtime Context (SRC) but, when in the context of processing a specific merge request, it can run under a specific access group because multiple applications can include the same data source.
Profile cache
The merge processor caches an instance of the profile so that it does not need to retrieve it from the database for every merge request – the profile provider cache expiration time DSS defines the profile caching window (default is 5 minutes). If any changes have been done to the profile in CPD, they are picked up the next time the merge processor refreshes the cache, and there is no need to restart the merge processor run to apply changes to the profile. As each data source is merged, the merge log is updated to reflect it in order for us to be able to determine what has and what has not been merge.
How does the merge processor state watcher work?
The merge process is a managed data flow run that is not resumed automatically by the data flow framework in case of failure. This is the technical reason behind the MergeProcessorStateWatcher implemented in Java code to periodically checks the state of the merge processor data flow run and, if the state is FAILED, resume the run.
The MergeProcessorStateWatcher check method is invoked by the Platform function pzMergeProcessorWatcher(), which is contained in the pzRunProfileMergeKafkaAlertCheck activity executed periodically by the pzCheckProfileMergeKafkaAlert job scheduler. By default, it runs every 3 minutes, and this can be configured by the merge processor state watched DSS (profile/merge/mergeProcessorWatcher). The Job Scheduler pzMergeMetricsConfigPulse also works in the same context for the same feature for reporting merge processor metrics.
We have created an internal BUG item (BUG-853551) to have the external documentation updated for these Job Schedulers – this is a work in progress and should be completed soon.
-------------------------------------
Pegasystems Inc.
US
@MarekG16680293 Our client is having same issues, starting to get these alerts after Infinity 23.1.1 update.
Job Scheduler pzMergeMetricsConfigPulse registration failed
Job Scheduler pzCheckProfileMergeKafkaAlert registration failed
Have you resolved these issues? If so, how?
Volksbank
PL
@Will Cho - Unfortunately the problem has not been fixed. We keep those 2 Jobs disabled on our primary application stack. when I checked last time, turning them off still resulted in a huge spike on util nodes CPU consumption.
Updated: 11 Sep 2024 8:06 EDT
Pegasystems Inc.
US
@MarekG16680293 BUG-853551 - i see a BUG item created internally and proposed in 24.2.
Issue primary reason description:
After updating the environment for staging from Pega version 8.6.6 to Infinity 23 Below error was occuring for the following Job schedulers pzMergeMetricsConfigPulse and pzCheckProfileMergeKafkaAlert PEGA0098 - Job Scheduler pzMergeMetricsConfigPulse registration failed. Error message: Invalid mapped node type RealTime PEGA0098 - Job Scheduler pzCheckProfileMergeKafkaAlert registration failed. Error message: Invalid mapped node type RealTime