StreamCruncher 1.05 Beta is out!

Ah, finally..I got the time to re-do the Partitioning and Pre-Filtering logic. Until this 1.05 release, Partitions had to pull Events from the source Stream/Table just before Query Processing - one step before the final Query could be executed. So, the overhead of fetching the Events and Pre-Filtering them ("where.. " clause in the Partition definition) would add some latency to the overall Query processing. Even though each Partition used to run in its own Thread, the whole process would have had to wait for all the Partitions to draw the new Events into their respective Partitions. From the 1.05 release, the Events are pre-fetched for each Partition by a separate group of Threads. This should improve the speed and CPU utilization on Multi-Core/Multi-Processor Hardware. Overall Latency should reduce noticeably on such Hardware.

As a consequence of this change in architecture, Partitions with the Pre-Filter clause do not trigger the Query unless the Events have passed through the Pre-Filter. In previous releases, unfiltered Events would trigger the Query (if the total Event Weight reached 1.0 or higher) an then would get filtered before reaching the Partition. This was very awkward for Queries with "New Events Windows", because Events that would trigger the Query spuriously would result in the "New Events Windows" to discard their contents.

Load distribution across multiple Queries is possible now without any untoward consequences (like the "New Events Windows" problem) because the Pre-Filtering works in a Thread pool of its own.