A simple guide to using Unix/GNU Linux command line tools for fiddling with log files (*runs on Windows too)

I've been meaning to write this post for years now. Every time I thought about compiling a basic list, I've told my self "Nah.. there must be tons of examples on the net". Yes there are tons of them but I couldn't find anything:

• That helped absolute noobs with a consolidated list
• That demonstrated actual fiddling with Java log files
• Something that works on Windows(!) No, I don't mean the awful Cygwin tool but something like Busybox or the wonderful Gow

So, here it is:
Enjoy!

Starting 2015 with yet another link dump

A belated happy new year! Here's some reading material I've been accumulating for a few months.

Distributed systems:

• http://grey-boundary.com/load-testing-apache-kafka-on-aws/ 
• http://www.bravenewgeek.com/dissecting-message-queues/ 
• http://www.slideshare.net/planetcassandra/cassandra-summit-2014-reading-cassandra-sstables-directly-for-offline-data-analysis
• All File Systems Are Not Created Equal: On the Complexity of Crafting Crash-Consistent Applications 
• Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems
• Apache Phoenix Transforming HBase into a Relational Database
• http://engineering.pinterest.com/post/104418761649/building-pinalytics-pinterests-data-analytics
• Druid @ Criteo

Performance related:

• Redis latency spikes and the Linux kernel: a few more details
• http://blog.kazuhooku.com/2014/11/the-internals-h2o-or-how-to-write-fast.html
• http://zeroturnaround.com/rebellabs/back-to-the-future-again-how-to-reduce-legacy-code-threats-before-they-happen/
• http://benmilleare.com/how-shaving-0-001s-from-a-function-saved-us-400-dollars/
• http://bad-concurrency.blogspot.in/2014/12/the-uncanny-valley-of-l3-cache.html
• Is it a CPU bound future?
• Choosing hardware to minimize latency
• http://yoshinorimatsunobu.blogspot.com/2014/03/why-buffered-writes-are-sometimes.html
• http://concurrencyfreaks.blogspot.in/2013/05/lock-free-and-wait-free-definition-and.html
• Yeah, this is what multi-threading is all about
• What happens when you type google.com ...
• What Programmers Say vs. What They Mean

On tuning:

• Multi-second ParNew collections but stable CMS
• https://support.datastax.com/entries/37991806-Some-tips-on-getting-the-most-IO-performance-out-of-SSDs
• http://blog.couchbase.com/often-overlooked-linux-os-tweaks
• https://tobert.github.io/post/2014-11-13-slides-disk-latency-and-other-random-numbers.html#/sysfs
• http://stackoverflow.com/questions/4101540/how-can-i-lower-the-weak-ref-processing-time-during-gc

Misc tech articles:

• Actor models - http://boundary.com/blog/2014/09/03/no-you-cant-have-a-pony/
• http://blog.griddynamics.com/2014/10/numeric-range-queries-in-lucenesolr.html
• http://stevewedig.com/2014/10/06/digraphs-dags-and-trees-in-java/
• http://martinsdeveloperworld.wordpress.com/2014/12/15/cjmx-a-command-line-version-of-jconsole/
• Strata of the Container Ecosystem 
• http://danluu.com/malloc-tutorial/

Formatting comments on Gerrit:

• https://wiki.openstack.org/wiki/GerritJenkinsGit#Gerrit_Comment_Formatting
• https://wiki.hpdd.intel.com/display/PUB/2012/02/16/Comment+formatting+on+Gerrit

That's it for now!

Fall 2014 tech reading

My posts are getting less frequent and when I do post something, I realize that they are mostly just links. Yes, work is keeping me busy.

 

Big data:

• Kafka/Hbase tuning
• Apache Tajo: Query Optimization Techniques and JIT-based Vectorized Engine
• "Jepsen II: Linearizable Boogaloo" by Kyle Kingsbury
• "Taming the 9s" by Ariel Weisberg
• Yahoo talks about Spark vs. Storm
• Druid / Building Real-time Systems with Open Source Technologies
• Raft, In Search of an Understandable Consensus Algorithm
• Storm's tuple tracking
• Cassandra 2.1 boot camp, Read/Write path
• Cassandra - Real Data Models of Silicon Valley
• Practice Makes Perfect: Extreme Cassandra Optimization
• Interesting Cassandra projects:

• https://github.com/rackerlabs/blueflood
• https://github.com/kairosdb/kairosdb
• https://github.com/OpenNMS/newts
• https://github.com/datastax/spark-cassandra-connector 
• http://prestodb.io/docs/current/connector/cassandra.html

Really? Another Hadoop SQL layer? Another Storm?

• https://github.com/KylinOLAP/Kylin
• http://tigon.io/
• http://phoenix.apache.org/
• http://incubator.apache.org/drill/
• http://samza.incubator.apache.org/ 
• http://prestodb.io/
• http://www.slideshare.net/ptgoetz/apache-storm-vs-spark-streaming

For those of you who knew about the original "column oriented stores" and "in-memory stream processing" - KDB - http://queue.acm.org/detail.cfm?id=1531242

Java:

• http://mishadoff.github.io/blog/effective-java/
• http://www.infoq.com/presentations/jvm-dynamic-optimizations
• Blocking vs. non-blocking
• AtomicXX lazySet
• Shenandoah: An open source pauseless GC for OpenJDK 

Java 8 - the good and ugly bits:

• I'm not looking forward to using Streams and CompletableFuture - JVM concurrency: Java 8 concurrency basics
• http://tododev.wordpress.com/2014/08/11/java-yield-like-using-stream-api/
• Worth the read: What's New in Java 8
• http://blog.informatech.cr/2014/04/04/jdk-8-interface-pollution/
• http://www.airpair.com/java/posts/parallel-processing-of-io-based-data-with-java-streams

Networks and systems:

• Hardcore tuning - http://10gigabitethernet.typepad.com/files/sf-110694-tc.pdf
• http://codearcana.com/posts/2013/05/18/achieving-maximum-memory-bandwidth.html
• http://labs.opendns.com/2014/10/01/redesigning-dns-database-low-latency/
• http://dtrace.org/blogs/brendan/2014/02/11/another-10-performance-wins/

The usual Scala and Go hate:

• http://www.slideshare.net/Bozho/scala-the-good-the-bad-and-the-very-ugly
• http://jeffknupp.com/blog/2014/08/19/go-is-fun-familiar-and-fast/
• http://stackoverflow.com/questions/1863460/whats-the-point-of-having-pointers-in-go
• Ha! "It's weird to see a young language getting stuck with inconsistent and strange syntax while still being a young language and claiming "historical" reasons"
• https://lobste.rs/s/ycor8k/why_go_is_not_good/comments/gs4pe9#c_gs4pe9
• http://yager.io/programming/go.html

Until next time!