Thursday, January 04, 2007

StreamCruncher 1.03 Beta is out! Changes include the "pinned" Aggregate Partition and the "entrance only" option for Aggregates. With this, the ReStockAlertTest has been modified to use the "Latest Rows/Events" Window instead of the "600 day Window", which in hindsight looks like it was not the right way to implement the Use Case.

I would've been thrilled to announce the support for ANTs Data Server and SOLID BoostEngine Databases. But the work involved to get those 2 DBs to work with SC was quite tiresome. So, yes StreamCruncher 1.03 Beta now supports SOLID BoostEngine and ANTs Data Server. That's a total of 7 Databases! Woo-hoo (Homer Simpson style)!

And if you want to be notified when there is a new release, I suggest you sign-up with FreshMeat and "Subscribe to new releases" at the bottom of their page.

However, there is a bug I've noticed in the Connection Pooling mechanism. SOLID integration also needed the Proxy classes just like how I had to for ANTs in my previous Post.

SOLID Driver has a bug in the ResultSet.getTimestamp(..) method. It throws an "Invalid Date" error. But the same column when retrieved through the getDate(..) or getTime(..) returns the corresponding Date/Time components. getString(..) on that column also works. So, I had to add a whole series of Proxies for the Connection, Statements (plain Statement, Prepared and Callable) and finally the ResultSet to intercept the getTimestamp(..) and getObject(..) calls on TIMESTAMP columns. After intercepting the call, I use the getString(..) method to retrieve the String form of the Timestamp, parse it back to java.sql.Timestamp and then return that to the caller. Mmmm..And obviously, because of all these Proxies, there is a 10-20 millisecond overhead for each method call. I might have to replace these Proxies with proper implementations and then forward the calls like in the GoF Adapter Pattern. That's for later, anyway.

And SOLID does not support the PreparedStatement.getParameterMetadata() method. It just hasn't been implemented.

But the biggest problem I've noticed here is the damn Connection leak! I noticed this while testing the SOLID DB integration. The SOLID installation I have is a Demo copy and so the number of Connections are limited. Even though the Test I was running used at most 2 simultaneous Connections, the Pool kept throwing a "Datasource rejected request. Too many connections" error. And then after adding a whole lot of Sys-Out calls I realised that the Pool never closes the Connections. D'oh! It just kept creating new Connections, which makes it unusable except for Demos, until it is fixed. I noticed this in ANTs and SOLID. I'm wondering if the Proxies I've used are causing this problem. Not sure if the same thing is happening in the other DBs. Arghh...more debugging work for this weekend. I suspect Apache DBCP or the Commons Pool.


knobboy said...

Same think happens with resultSet.getObject(..) method. The guess is that the driver keys off the column type which is timestamp, then uses the getTimestamp(..) method.