Graphs, machine learning, PostGres and other tidbits

I hadn't pushed out my "favorite reads of the season" for a while. So, here's a bunch of links to keep you occupied over the next few days.

Graphs, search and recommendations:

• Under the Hood: Building out the infrastructure for Graph Search
• LinkedIn's Cleo and Search (Cleo claims to have been inspired by FB's graph search)
• GraphLab - another take on large scale, distributed graph processing. Similar to Apache Giraph but not based on Hadoop code.
• Graph Based Recommendation Systems at eBay. Graphs, algebra and Cassandra. (I had to go back to basics to understand slide 9)

Statistics, machine learning presentations and resources:

• Data, Data, Data: Thousands of Public Data Sources
• Scalable and Flexible Machine Learning With Scala @ LinkedIn
• Big Data, Small Computers (H2O)
• A Practical Intro to Data Science
• Numeric Programming in Scala with Spire

A couple of peculiar, networking related blog entries.

 

Discussion on Redis mailing list about SSD / Twitter Fatcache / Facebook McDipper and a follow up.

 

While doing some research on NoSQL systems, especially Cassandra, I was surprised to hear that newer releases of Cassandra are moving away from the flexible, semi-structured column families. Instead with CQL, there is a well somewhat restrictive, repetitive schema that should work well for certain workloads. Is it me or does it look like NoSQL is grudgingly moving towards SQL?
 

Speaking of SQL, PostGres is moving in the other direction. Recent (9.x+) versions have some very interesting column data types - Array, HSTORE, JSON etc. Of course, its SQL support is obviously fantastic.

And finally, a nice talk on trade processing and a of paper on MongoDB for finance.


Ashwin.

Importing OpenSSL/EC2 .pem keypair to Java keystore

I spent several hours scouring the internet looking for a way to import (OpenSSL) Amazon EC2's .pem keypair into a Java keystore. At the end of this frustrating exercise I was baffled to see how scattered the information was.

(FYI, doing this on Windows, especially the OpenSSL interaction part, for self-signing a certificate was painful even with Cygwin. I had to resort to using my Linux distro running in a VM)

To save myself time in the future and for those of you tearing your hair out looking for the same information, here it is. (The file paths are not real. You have to clean them up to match your setup):

Here are my references in no particular order:

• http://stackoverflow.com/questions/2813843/possible-reasons-for-timeout-when-trying-to-access-ec2-instance 
• https://forums.aws.amazon.com/message.jspa?messageID=334402 
• https://confluence.atlassian.com/display/JIRA/Connecting+to+SSL+services
• http://www.agentbob.info/agentbob/79-AB.html
• http://www.startux.de/java/44-dealing-with-java-keystores
• http://codeartisan.blogspot.com/2009/05/public-key-cryptography-in-java.html
• http://wiki.eclipse.org/Generating_a_Private_Key_and_a_Keystore
• http://stackoverflow.com/questions/2484026/import-pem-public-and-private-keys-to-jks-keystore
• http://www.akadia.com/services/ssh_test_certificate.html

To complement this, there were other things I had to do (being a first time user of EC2) to make my EC2 instance accept SSH connections:

• http://blog.taggesell.de/?/archives/73-Managing-Amazon-EC2-SSH-login-and-protecting-your-instances.html
• http://dotnetbutchering.blogspot.com/2011/07/how-to-ssh-into-amazon-ec2-linux.html
• http://stackoverflow.com/questions/8193768/trying-to-ssh-into-an-amazon-ec2-instance-permission-error
• https://help.ubuntu.com/community/EC2StartersGuide#Official_Ubuntu_Cloud_Guest_Amazon_Machine_Images_.28AMIs.29
• https://www.openshift.com/page/install-and-setup-putty-ssh-client-for-windows

And then to install Oracle JDK 7 on my EC2 Ububtu image:

• http://thedaneshproject.com/posts/how-to-install-java-7-on-ubuntu-12-04-lts/

Ashwin.