Pega Thread Dump Format
Hello,
Using PRPC6 on Jboss, we got thread dumps from time to time, with a single custom format (not the one specified by Sun or IBM) :
2016-04-04 16:54:18,777 [x.x.x.x-8080-35] [ ] [ ] (.timers.EnvironmentDiagnostics) INFO x.x.x.x| Proprietary information hidden - ---
--- Thread Dump Starts ---
Thread[WorkManager(2)-13,5,JBoss Pooled Threads]
sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
…
Thread[ServerSocketRefresh,5,jboss]
java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:904)
--- Thread Dump Complete ---
I have looked to tools that can analyze these dumps without success. I tried IBM ISA and JCA tools. Also tried Samurai, no success.
Hello,
Using PRPC6 on Jboss, we got thread dumps from time to time, with a single custom format (not the one specified by Sun or IBM) :
2016-04-04 16:54:18,777 [x.x.x.x-8080-35] [ ] [ ] (.timers.EnvironmentDiagnostics) INFO x.x.x.x| Proprietary information hidden - ---
--- Thread Dump Starts ---
Thread[WorkManager(2)-13,5,JBoss Pooled Threads]
sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
…
Thread[ServerSocketRefresh,5,jboss]
java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.jboss.remoting.transport.socket.SocketServerInvoker$ServerSocketRefresh.run(SocketServerInvoker.java:904)
--- Thread Dump Complete ---
I have looked to tools that can analyze these dumps without success. I tried IBM ISA and JCA tools. Also tried Samurai, no success.
So I am looking to some specification or document that explains this Pega dump format to try figuring the dump out manually.
Have you please this kind of document ? Or can you suggest some working tools ?
Thank you very much for your help.
***Edited by Moderator Marissa to update categories***
**Moderation Team has archived post**
This post has been archived for educational purposes. Contents and links will no longer be updated. If you have the same/similar question, please write a new post.
Pegasystems Inc.
US
Not aware of any tools from pega. Generally we always recommend use the native thread dumps generated by the JVM vendor command/tools. Think of pega dump in the log as a notice: something may not be right (common case is performance), generate the JVM thread dumps for pega to analyze, using the tools you mentioned (IBM ISA is a great tool).
Sopra Group
FR
Thank you Kevin.
Your advice is helpful. Indeed, the Pega dump is proprietary. I was just asking if someone can explain the logic of the format Thread[WorkManager(2)-13,5,JBoss Pooled Threads] : what does 5 main, and what are the other infos for.
Thank you
Sopra Group
FR
Hi Vipin,
Thank you for your answer. Yes indeed, some helpful information is at the end of the dump. In our case, we can see the following ends (where for the first i have the thread reference immediatelly after dump end while for the second it can be found some lines later) :
1)--- Thread Dump Complete ---
2016-03-30 08:47:07,131 [-xxx-8080-3] [ ] [ ] (ngineinterface.service.HttpAPI) ERROR xxxx : com.pega.pegarules.pub.context.RequestorLockException
com.pega.pegarules.pub.context.RequestorLockException: Unable to synchronize on requestor xxx within 120 seconds: (thisThread = http-xxx-8080-3) (originally locked by = http-xxx-8080-1) (finally locked by = http-xxxd-8080-1)
2)--- Thread Dump Complete ---
2016-03-30 09:39:08,544 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxxx.Action) INFO - ;;;xxx
2016-03-30 09:39:08,585 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxxx.Action) INFO - ;;;xxx
2016-03-30 09:39:08,611 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxx.Action) INFO - xxx
2016-03-30 09:39:10,267 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxx.Action) INFO - ;;;xxx
2016-03-30 09:39:16,714 [xxx-8080-23] [ ] [ ] (ngineinterface.service.HttpAPI) ERROR xxx : com.pega.pegarules.pub.context.RequestorLockException
com.pega.pegarules.pub.context.RequestorLockException: Unable to synchronize on requestor xxx within 120 seconds:
Hi Vipin,
Thank you for your answer. Yes indeed, some helpful information is at the end of the dump. In our case, we can see the following ends (where for the first i have the thread reference immediatelly after dump end while for the second it can be found some lines later) :
1)--- Thread Dump Complete ---
2016-03-30 08:47:07,131 [-xxx-8080-3] [ ] [ ] (ngineinterface.service.HttpAPI) ERROR xxxx : com.pega.pegarules.pub.context.RequestorLockException
com.pega.pegarules.pub.context.RequestorLockException: Unable to synchronize on requestor xxx within 120 seconds: (thisThread = http-xxx-8080-3) (originally locked by = http-xxx-8080-1) (finally locked by = http-xxxd-8080-1)
2)--- Thread Dump Complete ---
2016-03-30 09:39:08,544 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxxx.Action) INFO - ;;;xxx
2016-03-30 09:39:08,585 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxxx.Action) INFO - ;;;xxx
2016-03-30 09:39:08,611 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxx.Action) INFO - xxx
2016-03-30 09:39:10,267 [ WorkManager(2)-92] [ STANDARD] [ xxxx] (xxx.Action) INFO - ;;;xxx
2016-03-30 09:39:16,714 [xxx-8080-23] [ ] [ ] (ngineinterface.service.HttpAPI) ERROR xxx : com.pega.pegarules.pub.context.RequestorLockException
com.pega.pegarules.pub.context.RequestorLockException: Unable to synchronize on requestor xxx within 120 seconds:
(thisThread = http-xxx-8080-23) (originally locked by = http-xxx-8080-31) (finally locked by = http-xxx-8080-31)
From this, I have idendified in the stack of threads which are were originating the trouble : http-xxxd-8080-1 for 1) and http-xxx-8080-31 for 2).
Rgds
Coforge Technologies
AU
Hi Halim,
The similar issue is happening for us.Can you please let us know fix for this Exception.
Thanks
Surya
Cognizant
IN
We are in the same boat. (Linux + Apache Tomcat + JRE 1.7.0 + PRPC 7.1.9 + Oracle 11g)
[LES ManagementDaemon] [STANDARD] [ ] (.timers.EnvironmentDiagnostics) INFO ---- Thread Dump Starts ---
Full Java thread dump with locks info "PegaRULES-Search[generic][T#153]" Id=14226 in TIMED_WAITING on lock=java.util.concurrent.SynchronousQueue$TransferStack@913e945
BlockedCount : 0, BlockedTime : -1, WaitedCount : 87, WaitedTime : -1
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
LEGATO HEALTH TECHNOLOGIES
IN
Hi,
we are also facing the same issue. can anyone can help us out y this is happening and what is the resolution for this issue.
"WorkManager(2)-81" Id=222 in WAITING on lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6b896b89 (running in native)
BlockedCount : 163, BlockedTime : -1, WaitedCount : 1199, WaitedTime : -1
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:184)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1998)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:410)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:958)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:736)
Locked synchronizers: count = 0
LEGATO HEALTH TECHNOLOGIES
IN
Nope, the issue is still there. we did not truncate the PR_SYS_STATUSNODES table. Before proceeding with that could you please be able to suggest why we should go for this approach and what was the root cause behind this thread dump.