Question
Rulesware LLC
MX
Last activity: 5 Jul 2021 5:17 EDT
Decision funnel explanation simulation test stop working
We are having issues running “Decision funnel explanation” a few days back it was running fine then it stop working but I am not able to see what is the issue, the Pega logs does not show any error.
I have tested the audience and strategies separately and those work fine, even create a quick DF that basically calls the DF_SimulationSegment for getting the audience segment and then pass the result to the strategy, it works fine.
We are having issues running “Decision funnel explanation” a few days back it was running fine then it stop working but I am not able to see what is the issue, the Pega logs does not show any error.
I have tested the audience and strategies separately and those work fine, even create a quick DF that basically calls the DF_SimulationSegment for getting the audience segment and then pass the result to the strategy, it works fine.
I am trying to find out the data flow id for when the simulation is created unlike if I create a batch process to run a data flow where the name of the data flow is in pyRuleName. Is the “decision funnel explanation” session test creating a data flow on the fly (Data-Decision-DDF-RunOptions.pxStartRun)? Or can I find what data flow is being invoked?
Similar, how can I review the ExplainDetails configuration to validate that there are no issues with it? I searched in Records->SysAdmin->Database Tables but is not there.
Notice that I open a similar question before but we already removed all the declare expressions we were using as part of the primary context: https://collaborate.pega.com/question/run-simulation-test-fails-when-created-form-cdh-portal-revision-management
Pegasystems Inc.
IN
Hello @FranciscoJavierG2301
When you say - We are having issues running “Decision funnel explanation” - what exactly happens? Does the simulation fail? Does it not start at all?
To answer a few questions that you have asked,
- Any simulation run (not just decision funnel explanation) is not directly associated with a data flow rule. It is managed by the engine.
- Due to the above, you will not be able to view a data flow rule associated with a simulation run.
- ExplainDetails again is a managed destination for decision funnel simulations.
It would help if you could specifically point out what the issue is, may be even include a screenshot or more describing the steps to reproduce.
-Thanks
Updated: 3 Mar 2021 16:00 EST
Rulesware LLC
MX
Hello @kamag
Thanks fo the quick response. Similar to my previous question it happens a few seconds after it start running:
"The run failed, because it exceeds the maximum number of failed record, which is currently set to 0. Details"
The details basically says the same:
"com.pega.dsm.dnode.impl.dataflow.resilience.policy.MaxFailedRecordsPerRunPolicy$ErrorThresholdReachedException: The run failed, because it exceeds the maximum number of failed record, which is currently set to 0"
I cannot find that exception in Pega Logs or DataFlow Logs. I am attaching the "Life Cycle Events".
How can I increase the log levels or improve the logging for the simulation? what other landing pages in Pëga can I use for problem determination?
Pegasystems Inc.
IN
Hello @FranciscoJavierG2301
The failure message on the run is pretty generic, which is usually the case. All that it tells us is there is an error somewhere in one of the components of the data flow run. For the actual stacktrace of the exception, we need to be able to access the component level metrics.
From your earlier post, we understand that you are probably on platform 8.5.1 and the component level metrics for simulation runs aren't accessible in this platform version. This is a known issue and is already patched in 8.5.2. Therefore, I reiterate that you should probably apply the 8.5.2 patch on your environment.
Any other suggestions @gongr or @BenKirby ?
Thanks!
Updated: 4 Mar 2021 15:52 EST
Pegasystems Inc.
NL
Hi @kamag, @FranciscoJavierG2301, I'm afraid I've spoken to @gongr on this and the only way to get at those data flow component metrics, and SEE why records are failing, is via the UI.
So 8.5.2 is the way to go, as you say, @kamag .
Thanks, Ben
Rulesware LLC
MX
Right now the metrics are secondary to us, what I want to find out is why the simulation stop working, as means for strategy testing over an audience.
What options do I have for doing troubleshooting in this scenario and find the reason?
Pegasystems Inc.
IN
The stage-level metrics are the best and fastest way to get to the root of the problem. "Failed records" on the stage level metrics show which records failed during the processing, where did they fail and why did they fail. I believe that is what we want to get to in this case as well - "why did they fail". Hence, the suggestion to apply the 8.5.2 patch.
A few other things you could validate are as follows.
- Check if you have at least one node under each of Decision Data Store, Adaptive Decision Manager, Data flow/Batch and Real-time Data Grid service in the NORMAL state. Given you are trying a decision funnel, the Real-time Data Grid service is important.
- Check if you are able to run the strategy through the same audience you used for the simulation above using a regular data flow run.
- Use the strategy test run panel to perform a batch run using the same audience as the problematic simulation. The batch run from the strategy test run panel is also a decision funnel simulation. If this fails, check if anything shows-up in the logs.
-Thanks!
Rulesware LLC
MX
Seems I found the issue, I have two when rules used as part of the Engagement Policies, lets call them "When A" and "When B" besides that I have also some other engagement policies using properties.
If I have following configuration the simulation runs (CONFIG_1)
- Eligibility: When A
- Applicability: Property condition A and Property condition B
- Suitability: When B
If I have following configuration the simulation does not run (CONFIG_2):
- Eligibility: When A
- Applicability: Property condition A and Property condition B and When B
In any of both configurations if I open the canvas for the group strategy and try to run the strategy with the test run panel I got following rule resolution error:
Seems I found the issue, I have two when rules used as part of the Engagement Policies, lets call them "When A" and "When B" besides that I have also some other engagement policies using properties.
If I have following configuration the simulation runs (CONFIG_1)
- Eligibility: When A
- Applicability: Property condition A and Property condition B
- Suitability: When B
If I have following configuration the simulation does not run (CONFIG_2):
- Eligibility: When A
- Applicability: Property condition A and Property condition B and When B
In any of both configurations if I open the canvas for the group strategy and try to run the strategy with the test run panel I got following rule resolution error:
Failed to execute [NBA_MyIssue_MyGroup_E_Customer].[PropFilter___Result___Final___Output] from [NBA_MyIssue_MyGroup_Customer]. Possible components [[NBA_MyIssue_MyGroup_Customer].[Applicability],[NBA_MyIssue_MyGroup_Customer].[Suitability],[NBA_MyIssue_MyGroup_Customer].[Offer Collector],[NBA_MyIssue_MyGroup_Customer].[NBA_MyIssue_MyGroup_E_Customer],[NBA_MyIssue_MyGroup_Customer].[NBA_MyIssue_MyGroup_S_Customer],[NBA_MyIssue_MyGroup_Customer].[Best Result],[NBA_MyIssue_MyGroup_Customer].[MyGroup Extension],[NBA_MyIssue_MyGroup_Customer].[Eligibility],[NBA_MyIssue_MyGroup_Customer].[Strategy___Result___Final___Output],[NBA_MyIssue_MyGroup_Customer].[NBA_MyIssue_MyGroup_A_Customer]]: Cannot find rule {pxObjClass=Rule-Obj-When, pyClassName=@baseclass, pyBlockName=WhenA}
Some final facts:
- The When A and When B rules are in Artifacts:01-01-05, same ruleset as the strategy.
- Both when rules are marked as relevant records so are visible in the NBA-D
- The simulation issue happens on Dev only; Run a simulation with CONFIG_2 runs ok on QA
- On QA I get the same rule resolution error running the strategy from test run panel
Any idea/suggestion about:
- why CONFIG_2 fails?
- why the strategy does not recognize the when rules?
Best regards
Javier
Pegasystems Inc.
IN
Hello @FranciscoJavierG2301
Thank you for the extremely detailed analysis.
I can't think of a reason on the top of my mind why the same set-up should behave differently in two different environments unless it is something to do specifically with the environment. A run-time exception like that on the strategy rule can possibly also surface during say a Real-time Container (RTC) call and not just when running a simulation alone.
Therefore, I'd suggest you to create an incident so that an appropriate team could take a closer look.
-Thanks!
Verizon
IN
Can you please help with the 8.5.2 patch, How can i know if it is already installed in my Pega CDH 8.5 instance already ?
Or do you have any recommended way to get that installed. I am facing similar problem.
--
Thank you
Verizon
IN
I would like to have your inputs here, Based on our last conversation in 8.5.2 Decision Funnel Destination is no more a Database table.
We were trying to see that is action, but could not do so. At this stage:
1. all the DSM Service on nodes are working fine
2. We are running simulation from Strategy Canvas for Batch Run
3. Selecting a Data Set with 1000 records
4. The Simulation run does not have any error after completion
5. All the reports are created with 0 instances.
--
Thank You
Prasanna Kumar
Pegasystems Inc.
GB
The version you are coming from did not use VBD for aggregating numbers, in 8.5.2 you will now be using VBD. You say the RTDG service is running fine?
Verizon
IN
Yes Alex, We are using 8.5.2 and RTDG Service is running fine on this.
We could run the simulation but do not see any records in reports generated by the Simulation Run.
--
Thank You
PK
Pegasystems Inc.
GB
@PK2013137 do you see numbers on the canvas when you select the simulation run in test run panel?
Verizon
IN
Yes Alex, We could see 1000 customer count while selecting Data Set for Simulation. Also after completion of Simulation Run we could see 1000 records processed.
After clicking on "More Details" i could see DF has 4600 records finally for Destination, but again no records in RD's.
--
Thank You
PK
Pegasystems Inc.
GB
Sorry, I mean select the simulation on the test run panel on your strategy you run the simulaiton on. Click the components and see if you have results, navigate the strategy (including sub-strategies) to see if your simulation has results at any level.
Verizon
IN
I ran simulation for a simple strategy (1 Prop Import + Filter + Property Set + Result), Now my data set has just 1 Customer record, From Test Run Panel :
1. I am doing Single Case for this Data Set - I am getting results
2. Now I am doing Batch for "New Simulation Test" selecting same data set : In Simulation LP I see the Processed record count as 1 also DF run doe not have any error , I do not see the proper details of where the records are stored in destination. Now coming back to Strategy Canvas, after selecting same Simulation Run which i ran just now. I do not see any results at component levels.
3. Step 2 is repeated for Data Set having 1000 records, I could see same outcome.
--
Thank you
PK
Pegasystems Inc.
GB
@PK2013137 I think best to raise an incident for this issue as pretty tricky to understand what is happening via email. Pulse the incident number here when raised and GCS will help with initial diagnosis and forward to engineering team if required.