How to avoid message "Unable to unlock work object"?
This is a technical question, as well as a support process question.
- Consider the case where a user has accidentally left a work item locked, and then logged in to a new node, and try to open that work item.
- Going back as far as I remember (v3), the user gets the message about it being locked by them, and has a button to click to try and release that.
- In v7, we're seeing the screen say "Unable to unlock work object" - and yet, if the user ignores this message and clicks the link to try and Perform again, it will work.
(true that in v7.1.5, part of the message on the on the first screen is that the lock won't release right away... and yet the "unable to unlock work object" message will trump that to most users.)
We've filed a couple of SR's on this -- we suspected that the solution may lie in disabling v6.3+'s lockCache. So that's what we've done.
Now here's the process twist on it -- looking through the Support Articles, I see that this question has been asked 3 times recently, and 3 different answers are listed here.
Grant that there may be subtle difference between these -- one references v7 -- though note SA-5342 speaks of a HFix, but as we all know, HFix'es aren't quite regularly supported in v7. So as a customer, searching through this today, I would probably simply ask via SR, can I get this same HFix?
This is a technical question, as well as a support process question.
- Consider the case where a user has accidentally left a work item locked, and then logged in to a new node, and try to open that work item.
- Going back as far as I remember (v3), the user gets the message about it being locked by them, and has a button to click to try and release that.
- In v7, we're seeing the screen say "Unable to unlock work object" - and yet, if the user ignores this message and clicks the link to try and Perform again, it will work.
(true that in v7.1.5, part of the message on the on the first screen is that the lock won't release right away... and yet the "unable to unlock work object" message will trump that to most users.)
We've filed a couple of SR's on this -- we suspected that the solution may lie in disabling v6.3+'s lockCache. So that's what we've done.
Now here's the process twist on it -- looking through the Support Articles, I see that this question has been asked 3 times recently, and 3 different answers are listed here.
Grant that there may be subtle difference between these -- one references v7 -- though note SA-5342 speaks of a HFix, but as we all know, HFix'es aren't quite regularly supported in v7. So as a customer, searching through this today, I would probably simply ask via SR, can I get this same HFix?
|
Article |
Version |
Last Updated |
Response |
|
SA-5783 |
6.3 sp1 |
Jan 29, 2015 10:33:45 PM |
Wait for the system pulse to logged out [sic] the other session. |
|
SA-6795 |
6.3 sp1 |
Feb 28, 2015 9:22:36 AM |
With below entry in prconfig.xml and restarting server, reported behaviour was resolved. <env name="database/lockcache/enabled" value="false"/> |
|
SA-5342 |
7.1.6 |
Jan 20, 2015 10:53:23 AM |
HFix-20494 was given to remove the error message "Unable to unlock work object". |
|
7.1.5 |
Dec 5, 2014 4:04:46 PM |
Submitted product enhancement request item FDBK-9861 |
Pegasystems
US
Hi Jon,
Until recently, a confusing aspect of the unlocking mechanism was that the code would tell the other node's pulse to delete the lock-holding requestor, and the first node would immediately try to acquire the lock again which often fails since the pulse only runs every 60 seconds, so another failure would be announced.
A recent improvement is that after the code tells the other node's pulse to delete the lock-holding requestor, the review harness is re-shown. The user can then click the action link on the harness to try to acquire the lock. If the lock is still not available, the process repeats. From the user's point of view, the review harness may be shown several times before the pulse finally runs and the action link is able to lock the work.
I'll let someone else chime in and tell us exactly which base level of Pega 7 this improvement went in to. /Eric
BNY Mellon
US
Eric - good to be working with you again!
re: "Until recently, a confusing aspect of the unlocking mechanism was that the code would tell the other node's pulse to delete the lock-holding requestor, and the first node would immediately try to acquire the lock again which often fails since the pulse only runs every 60 seconds, so another failure would be announced."
I assume this is using the v6.3+ lockCache? Since prior to that, locks were maintained in the DB directly. And I was wondering if turning off lockCache is an easy way to restore this (or only in v6.3)
Maybe looking up HFix-20494 will provide a hint?
Pegasystems Inc.
US
Jon,
Let me touch on a couple of your points. Yes, turning off the caching added in 6.3 will remove this situation. To the best of my knowledge the same system setting from 6.3 should work, though I've never actually done it in Pega 7. You will miss out on the benefits that the functionality provides (mostly performance, I believe). The way the caching is designed to work, you need to wait for the system pulse to propagate any change to all nodes when you are unlocking via the page we are talking about. It looks like that didn't always happen and that's what resulted in the HFix you found. If I understand the notes correctly, installing the HFix will still require the system pulse, so wouldn't ultimately address your initial concern. That said, if you are on Pega 7.1.6, by all means feel free to enter an SR to request the HFix. HFixes are totally supported in Pega 7, we just try and avoid them, preferring to migrate customers/fix things in the latest ML. That prevents the sorts of issues that you can find in the 6 series and earlier, where the HFix you want has a large number of dependencies which can introduce unintended consequences.
Thanks,
Mike
BNY Mellon
US
We're still on ML5, alas. And, in general, we've been Is HFix-20494 included in ML7?
I want to get beyond the "feeling the elephant" narrow-scoped approach we take at times.
What exactly does HFix-20494 supply? Is it that the "recent improvement" Eric was talking about? If the solution is to refresh the harness to get the lock status... why not just support an Ajax call to check the status of the lock?
(and I heard that in ML8 this is moving to a new approach for managing the locks -- what is available there?)
Pegasystems Inc.
US
Based on the dates, I think HFix-20494 missed ML7 and is probably in ML8. I'm not sure if this is the change Eric mentioned. The HFix notes state they "Need to add custom message which will say this option is not effective immediately." Adding that message may also include showing the harness again, as Eric mentions. If there are plans to revisit the locking code in ML8, I'm not familiar with them, so I can't comment on if/how that will be implemented.
Updated: 5 Mar 2015 16:21 EST
Pegasystems Inc.
US
Jon:
My tracing (in internal tracking systems) of the case objects you cited for HFix-20494 shows that this issue was Found in Pega 7.1.6 and Fixed in Pega 7.1.8 -- not yet GA.
When Pega 7.1.8 is GA, you will find the 7.1.8 RelNote for the bug presented with the SR references as ISSUE:186503 in the section, Issues Addressed in This Release.
~Mary
BNY Mellon
US
Thanks to you both. I am trying to zero in on the documentation issue, though. When I search the Support Articles, I got the 3 articles above, suggesting different solutions. I'd think it sensible to collapse them all into one article, and describe the different reasons to try each solution.
I had questions related to lock management on thread now -- when I can access that again, I may post as a separate. But what I'm after here is a single document explaining case locking from soup to nuts -- issues, features, and all.
Ironically, one of my current pet peeves with GCS Support portal is the locking. As a customer, all I can do is add a comment or an attachment ... shouldn't necessarily require a case lock for that. One of my open SR's has been locked for the last couple of hours, apparently.
Pegasystems Inc.
US
Jon:
Thanks for emphasizing the multiple SA issue (one problem - multiple SAs with different solutions).
Before we piloted the SAs last spring, Marc Dumas identified this scenario and the issue.
The decision to allow multiple (redundant or variant) SAs was based (if I remember correctly) on the following reasons:
- Customer vetting (Your Rating and This Solved My Problem) would designate the most authoritative article. We want C2C promotion of content for self-service.
- GCS customer support engineers search the PegaSupport application as their first target knowledge base when handling customer cases. They should find existing SAs and cite them when resolving their SRs. This best practice would reduce the number of SA DUPs.
I will relay your excellent suggestion of merging multiple SA content into one master SA to the GCS Leadership Team.
Thanks!
~Mary
BNY Mellon
US
Incidentally, we've set database/lockcache/enabled to false, and on our QA system we're not seeing consistent release of the locks. We may need to open a new SR at this point. My worry is that LockManagerImpl is not logging enough to debug() to help pinpoint the problem.
If we are forcing another session to be closed, and that session is releasing locks, that that should be logged, at least at DEBUG level.
Regarding HFix-20494, I'm curious what the complexity of the fix is. Is this something that could be installed in ML5 -- or, I suppose, we can just cover that in the SR.
Pegasystems Inc.
US
It sounds like you may have reached the point where an SR makes sense to review the system setting in Pega 7. If we need to add additional logging, that can be looked at as part of the SR. Along the same lines, reviewing the feasibility of backporting the other SR could be considered.
Pegasystems Inc.
US
Jon,
If you do end up logging an SR, can you please mention that you used the community before logging the SR?
As this is a pilot, we don't haven't made any formal updates to My Support Portal, but that is one of the things we are assessing.
Given the content manager options currently available, I am going to mark this as Assumed Answered to pull it off the Open Questions board. I'll look into other options for better indicating a discussion results in an SR. Also, if the SR results in a Support Article, it might be good to come back and reference it in this thread and mark that post as the Correct Answer.
Updated: 10 Mar 2015 11:05 EDT
Pegasystems Inc.
US
Jon,
If you do submit an SR, I assume that you will cite the three SAs you mentioned above.
When resolved, the SR can spin a KR for a PDN Article that refers to and consolidates the scenarios and info of the three existing SAs (5342, 5783, 6795) along with new information of your resolved case.
Then we'd have the master article (PDN Article) for the issue.
Please suggest the KR for a PDN Article in your SR description: Thanks!
-Mary
BNY Mellon
US
Ok, I'm still a bit perplexed about how lock management works.
The button "End Other Sessions to Release Lock" calls the activity ReleaseLock, which calls the activity WorkUnlock (with parameter differentRequestor=true), calls the API function LockManager.unlock() will release the locks in the following way:
A) Via direct SQL calls (the legacy way)
B) Stopping the other requestor (the new way)
If lockCache is disabled, which method is used? I had assumed (A), but we see that (B) is happening, that the other requestor is stopped.
Either way, this would be a handy Ajax function, naturally. And we'd want an infoForced to write to the logs such. (leading to my foray here - When are passivated requestors restored?)
Also -- isn't stopping the other requestor a wee bit out of bounds for a humble function like unlock()...? I mean, might WorkUnlock be called by other callers along the way? (e.g., there's 32 activities that call it, but most probably set differentRequestor=false, the default)
FWIW, here's the sequence of logs (trimmed much) of what we see happens when we try to release. As above, with lockCache disabled, it looks like it's trying to delete it directly.
15:05:27,337 (LockManagerImpl) DEBUG - Attempting to unlock "MY-WORK CAM-138" on any Requestor
Ok, I'm still a bit perplexed about how lock management works.
The button "End Other Sessions to Release Lock" calls the activity ReleaseLock, which calls the activity WorkUnlock (with parameter differentRequestor=true), calls the API function LockManager.unlock() will release the locks in the following way:
A) Via direct SQL calls (the legacy way)
B) Stopping the other requestor (the new way)
If lockCache is disabled, which method is used? I had assumed (A), but we see that (B) is happening, that the other requestor is stopped.
Either way, this would be a handy Ajax function, naturally. And we'd want an infoForced to write to the logs such. (leading to my foray here - When are passivated requestors restored?)
Also -- isn't stopping the other requestor a wee bit out of bounds for a humble function like unlock()...? I mean, might WorkUnlock be called by other callers along the way? (e.g., there's 32 activities that call it, but most probably set differentRequestor=false, the default)
FWIW, here's the sequence of logs (trimmed much) of what we see happens when we try to release. As above, with lockCache disabled, it looks like it's trying to delete it directly.
15:05:27,337 (LockManagerImpl) DEBUG - Attempting to unlock "MY-WORK CAM-138" on any Requestor
15:05:27,338 (LockManagerImpl) DEBUG - Requestor HDF3E51BF2B741CED14B36FC8EBEEF1D5 running following SQL to quick delete lock "MY-WORK CAM-138"...:
15:05:27,341 (LockManagerImpl) DEBUG - Requestor HDF3E51BF2B741CED14B36FC8EBEEF1D5 running following SQL to check for lock "MY-WORK CAM-138"...
15:05:27,716 (LockManagerImpl) DEBUG - Attempting to check lock for ID: MY-WORK CAM-138
15:05:27,717 (LockManagerImpl) DEBUG - Requestor HDF3E51BF2B741CED14B36FC8EBEEF1D5 running following SQL to check for lock "MY-WORK CAM-138":...
15:05:27,721 (LockManagerImpl) DEBUG - Requestor "HDF3E51BF2B741CED14B36FC8EBEEF1D5" found lock on "MY-WORK CAM-138" owned by "H8EADD373A10ECCF2FB274C17AF5F10BF"
15:05:27,722 (LockManagerImpl) DEBUG - Lock not owned by requestor HDF3E51BF2B741CED14B36FC8EBEEF1D5
BNY Mellon
US
This is really amazing... I posted this originally on March 4th.
And I now see the fourth different answer in the Support Article catalog.
Actually, fourth and fifth.
For a user using version 7.1.6, two suggested resolutions:
- From Developer Portal, unlock the cases
- In the database delete from pr_sys_locks table -- as discussed above, not supported in the lockCache error.
This is really frustrating to see this -- even after I raised visibility into this very issue!
Pegasystems Inc.
US
Jon,
I acknowledge your frustration and appreciate your input. We recognize customer self-support and community are critical components of the digital age. By providing these mechanisms, we provide the means for people to voice frustrations such as yours in a constructive way.
Support Articles are a crucial piece of our overall vision. They, much like this pilot, are something we have decided to provide as an offering now, with a commitment to continually improve upon them. These improvements may not happen overnight, so please bear with us. In the meantime, know that your input has been heard and I thank you for it.
Thanks.
B.
Pegasystems Inc.
GB
Support Articles are based on resolved individual SRs, so I guess it is not that surprising there will be duplicates and slightly different solutions (some relevant to different versions of PRPC), as what works / is acceptable in one case may not be in another.
My current take on it is that they are not intended as definitive answers to issues per se (in the same way that a formal PDN article might be). If from a variety of SAs an acceptable solution can be identified without having to raise an SR then great, they have done their job.
Previous to SAs you might have raised an SR and we would have searched this same information internally and provided advice based on previous cases. Now that information is available externally and hopefully can provide useful. As Mary Carbonara mentioned the "Your Rating and This Solved My Problem" functions will go some way to identifying the SAs that have proved more useful.
That said we definitely need to be aware of duplicates and contradictions that will inevitably arise so hopefully they will evolve over time.
Pegasystems Inc.
US
Jon,
Did you consider my advice posted previously in this thread?
Submit an SR requesting a KR for a master PDN Article that would consolidate the various customer case scenarios and the refine the info provided by the incremental set of SAs on this topic. The master PDN Article on this topic would be updated to reflect additional scenarios and more targeted guidance on product behavior and use.
Marc Alderman, in his most recent reply here, is absolutely correct in clarifying the objective of Support Articles (SAs) and how they differ from PDN Articles. This distinction, I believe, argues for an SR asking for a KR for a PDN Article on this topic.Your SR would cite the collection of SAs and links to unanswered questions posted in this community and elsewhere.
-Mary
BNY Mellon
US
This is probably worth a spunoff-thread by now. I did file an SR because we still had a problem validating this. And I did mention, as part of that follow-up dialogue, about the different SA's. But I wasn't as specific as you asked -- "Submit an SR requesting a KR for a master PDN Article..." -- I feel like that's beyond my scope.
Pegasystems Inc.
US
Jon:
Your original problem (for which you submitted an SR) is being addressed by two product enhancements:
These FDBK items complement FDBK-9861, listed above in your table of Support Articles (SAs).
-Mary