If you find yourself trying to analyze a big heap dump (20-30GB) downloaded from your production server to your staging/test machines.. only to find out that X-over-SSH is too slow then this article is for you.
As of Nov 2013, we have 2 options - Eclipse MAT and a hidden gem called Bheapsampler.
Option 1:
Eclipse Memory Analyzer is obviously the best tool for this job. However, trying to get the UI to run remotely is very painful. Launching Eclipse and updating the UI is an extra load on the JVM that is already busy analyzing a 30G heap dump. Fortunately, there is a script that comes with MAT to parse the the heap dump and generate HTML reports without ever having to launch Eclipse! It's just that the command line option is not well advertised.
Command line heap analysis using Eclipse MAT:
Assuming
Eclipse MAT is installed and we are inside the mat/ directory, modify MemoryAnalyzer.ini heap settings to use a large heap to handle large
dumps:
-startup
plugins/org.eclipse.equinox.
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.
-vmargs
-Xms24g
-Xmx24g
Run MAT against the heap dump:
./ParseHeapDump.sh ../today_heap_dump/jvm.hprof
This
takes a while to execute and generates indices and other files to make
repeated analysis faster. Then use the indices created in the previous
step and run a "Leak suspects" report on the heap dump.
./ParseHeapDump.sh ../today_heap_dump/jvm.hprof org.eclipse.mat.api:suspects
The
output is a small and easy to download jvm_Leak_Suspects.zip. This
has HTML files just like the MAT Eclipse UI. It can be easily
SCP'ed/emailed around.
Other report types possible.
org.eclipse.mat.api:suspects
org.eclipse.mat.api:overview
org.eclipse.mat.api:top_
More details - http://wiki.eclipse.org/index.php/MemoryAnalyzer/FAQ.
Option 2:
http://dr-brenschede.de/bheapsampler is something I chanced upon. It is a sampling heap dump reader and so it works for very large heap dumps where MAT sometimes fails. Being a sampling reader, the output is also a little imprecise but helps a great deal when you have nothing else. The tool seems to be closed source and is very sensitive to heap dump corruptions.
As an aside, here's something that might be useful for the initial heap dump quickly - https://blogs.atlassian.com/2013/03/so-you-want-your-jvms-heap/.
Sunday, November 24, 2013
Analyzing large Java heap dumps when Eclipse Memory Analyzer (MAT) UI fails
Subscribe to:
Post Comments (Atom)
5 comments:
Sounds great... except when I try it (ssh'd through to bash on a linux box), I get "Unable to initialize GTK+". Sounds like it really wants to start up some kind of UI. Any thoughts?
I haven't tried it on Linux in a while. I think your best bet would be to post on the Eclipse MAT mailing list.
The problem seems to be that the (newer?) versions of ParseHeapDump.sh call the MemoryAnalyzer program directly, which in turn expects GTK+ to be there.
I made my own version of ParseHeapDump.sh that does this instead:
/tools/java/java/bin/java -Xmx16g -Xms16g \
-jar plugins/org.eclipse.equinox.launcher_1.3.100.v20150511-1540.jar \
-consoleLog -consolelog -application org.eclipse.mat.api.parse "$@"
And that seems to work fine. (Substitute the location of your own 'java' for the /tools/java... above.)
Thanks for posting the solution!
Use "Memory Analyzer 1.0.0". It works perfect as described by Ashwin.
Post a Comment