Thursday, April 07, 2011

The little gem that is BusyBox (for Windows)

As a Windows user (no shame) I have, for years searched for a simple GNU-like toolkit - grep, awk, tail and other such goodies enjoyed by Linux users. Yes, there's Cygwin but it's a beast - too big and a pain to install. Sometimes, I've even resorted to starting a Linux VMWare image just to run a simple awk script to munge some log files.

But today...today I found BusyBox and there's a compact 600KB BusyBox exe for Windows! What does it have? Well.. what does it not have?! It has all the essentials:

[, [[, ar, ash, awk, base64, basename, bash, bbconfig, bunzip2, bzcat,
bzip2, cal, cat, catv, cksum, cmp, comm, cp, cpio, cut, date, dc, dd,
diff, dirname, dos2unix, echo, ed, egrep, env, expand, expr, false,
fgrep, find, fold, getopt, grep, gunzip, gzip, hd, head, hexdump, kill,
killall, length, ls, lzcat, lzma, lzop, lzopcat, md5sum, mkdir, mv, od,
pgrep, pidof, printenv, printf, ps, pwd, rm, rmdir, rpm2cpio, sed, seq,
sh, sha1sum, sha256sum, sha512sum, sleep, sort, split, strings, sum,
tac, tail, tar, tee, test, touch, tr, true, uncompress, unexpand, uniq,
unix2dos, unlzma, unlzop, unxz, unzip, usleep, uudecode, uuencode, vi,
wc, wget, which, whoami, xargs, xz, xzcat, yes, zcat
This is where I'd use it the most - to compress and summarize JVM thread dumps for quick analysis. It's based on JPMP, but here's my version of it. First, download BusyBox for Windows and then use the simple script below to munge your JVM thread dump.

This is the first part of the script:
This is the awk pattern that is used by the script above.
All you have to do now is run it against your thread dump file:
jpmp.bat ..\jstack.out > ..\jstack.out.log

And it converts a huge file like this...
2011-04-07 20:44:06
Full thread dump Java HotSpot(TM) 64-Bit Server VM (16.2-b04 mixed mode):

"TimerQueue" daemon prio=6 tid=0x000000004ec33000 nid=0x1268 in Object.wait() [0x000000004a3cf000]
   java.lang.Thread.State: WAITING (on object monitor)
 at java.lang.Object.wait(Native Method)
 - waiting on <0x0000000034004608> (a javax.swing.TimerQueue)
 at javax.swing.TimerQueue.run(TimerQueue.java:232)
 - locked <0x0000000034004608> (a javax.swing.TimerQueue)
 at java.lang.Thread.run(Thread.java:619)

"D3D Screen Updater" daemon prio=8 tid=0x000000004af1e000 nid=0x186c in Object.wait() [0x000000004f40f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
 at java.lang.Object.wait(Native Method)
 - waiting on <0x0000000034e16ba8> (a java.lang.Object)
...
...
...
...
"Reference Handler" daemon prio=10 tid=0x0000000000823800 nid=0x18e4 in Object.wait() [0x000000004940f000]
   java.lang.Thread.State: WAITING (on object monitor)
 at java.lang.Object.wait(Native Method)
 - waiting on <0x0000000034dda018> (a java.lang.ref.Reference$Lock)
 at java.lang.Object.wait(Object.java:485)
 at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
 - locked <0x0000000034dda018> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=10 tid=0x0000000000820000 nid=0x14c4 runnable 

"GC task thread#0 (ParallelGC)" prio=6 tid=0x0000000000777800 nid=0x173c runnable 

"GC task thread#1 (ParallelGC)" prio=6 tid=0x0000000000779800 nid=0x4b8 runnable 

"GC task thread#2 (ParallelGC)" prio=6 tid=0x000000000077b000 nid=0x15d0 runnable 

"GC task thread#3 (ParallelGC)" prio=6 tid=0x000000000077c800 nid=0x199c runnable 

"VM Periodic Task Thread" prio=10 tid=0x0000000049580800 nid=0x1984 waiting on condition 

JNI global references: 1355

Into a tidy summary like this. Very useful if you have thread dumps from 20 servers taken every 30 seconds.
All this without leaving the comfort of your Windows system!
2 j.lang.Object.wait,j.io.PipedInputStream.read,j.io.PipedInputStream.read,sun.nio.cs.StreamDecoder.readBytes,sun.nio.cs.StreamDecoder.implRead,sun.nio.cs.StreamDecoder.read,j.io.InputStreamReader.read,j.io.BufferedReader.fill,j.io.BufferedReader.readLine,j.io.BufferedReader.readLine,sun.tools.jconsole.OutputViewer$PipeListener.run
1 sun.awt.windows.WToolkit.eventLoop,sun.awt.windows.WToolkit.run,j.lang.Thread.run
1 j.lang.Object.wait,sun.java2d.d3d.D3DScreenUpdateManager.run,j.lang.Thread.run
1 j.lang.Object.wait,jx.swing.TimerQueue.run,j.lang.Thread.run
1 j.lang.Object.wait,j.lang.ref.ReferenceQueue.remove,j.lang.ref.ReferenceQueue.remove,sun.java2d.Disposer.run,j.lang.Thread.run
1 j.lang.Object.wait,j.lang.ref.ReferenceQueue.remove,j.lang.ref.ReferenceQueue.remove,j.lang.ref.Finalizer$FinalizerThread.run
1 j.lang.Object.wait,j.lang.Object.wait,sun.awt.AWTAutoShutdown.run,j.lang.Thread.run
1 j.lang.Object.wait,j.lang.Object.wait,j.lang.ref.Reference$ReferenceHandler.run
1 j.lang.Object.wait,j.lang.Object.wait,j.awt.EventQueue.getNextEvent,j.awt.EventDispatchThread.pumpOneEventForFilters,j.awt.EventDispatchThread.pumpEventsForFilter,j.awt.EventDispatchThread.pumpEventsForHierarchy,j.awt.EventDispatchThread.pumpEvents,j.awt.EventDispatchThread.pumpEvents,j.awt.EventDispatchThread.run

Until next time!

9 comments:

gway dust said...

isn't it simpler to just use gnuwin32

Ashwin Jayaprakash said...

I had never heard of gnuwin32. Thanks!

Anonymous said...

Or maybe good old Cygwin

Shiraz Kanga said...

Ashwin, I also love BusyBox and it's tiny size. Here is a tip to make things easier for you on Windows.

Instead of running:

busybox awk -f jpmp_strace.awk < %1 | busybox sort | busybox uniq -c | busybox sort -rnk1

I prefer to have:

awk -f jpmp_strace.awk < %1 | sort | uniq -c | sort -rnk1

You can easily achieve this. Just create a .bashrc file in the same location as busybox.exe and create an alias for every busybox command.

Here is what mine looks like:


#!./busybox bash

# Don't allow any other versions of these commands on the PATH
export PATH=""

# Setup aliases for all the available busybox applets
alias [="./busybox ["
alias [[="./busybox [["
alias ar="./busybox ar"
alias ash="./busybox ash"
alias awk="./busybox awk"

....... Stuff deleted ......


alias xzcat="./busybox xzcat"
alias yes="./busybox yes"
alias zcat="./busybox zcat"

Ashwin Jayaprakash said...

@Shiraz That's a nice idea!

Ashwin Jayaprakash said...

Gow (https://github.com/bmatzelle/gow/wiki) - unstable but promising.

Bilal Malik said...

What exactly do I need to download to run BusyBox on windows. I went on this link below and I found busybox.tar.gz files and not a .exe for windows.
http://busybox.net//downloads/
Please help

Ashwin Jayaprakash said...

Use the busybox for win32 link above.

adampasz said...

You can install with scoop. https://github.com/lukesampson/scoop