Colin Clark

I’ve spent the last couple of days, after absorbing the Sybase acquisition of Aleri’s assets, looking at the various CEP players websites for evidence of cloud (grid name du jour) deployment. I haven’t found anything worth mentioning.  That’s not to say that I might have missed something.  But my guess is that with all the economic woes lately, that the CEP vendors have either chosen to ignore or just simply aren’t aware that the rest of the world is actually solving real problems with cloud deployments. Which is interesting. It’s interesting because one of the pioneers in CEP & database space, Mike Stonebreaker (love that name), has gone on to work in stealth mode (it says stealth on their web site anyway) over at VoltDB. VoltDB is based upon work done on HStore and would appear, on the surface anyway, to incorporate all of the things we’d expect in an OTP environ... (more)

Erlang, RabbitMQ, & Redis

VMWare has been on a buying spree lately. In the last month, they’ve announced both Redis and RabbitMQ. Here’s VMware’s take on Redis, and spring source’s on RabbitMQ.  RabbitMQ is built with Erlang. Much Rejoicing in the Village We use both of these technologies at Cloud Event Processing.  And we love Erlang too.  VMware’s acquisition of these technologies not only validates our decisions, which we are very selfishly pleased about, but also sends an interesting message. The Message Please Everyone’s busy abstracting resources in the cloud – making resources like compute, stora... (more)

Why CEP in the Cloud Makes Sense

CEP isn’t really about low latency.  The ability to do things quickly is important, just as in any system – especially those systems that grow and need to handle a lot of information.  Doing things quickly means doing things efficiently.  And doing things efficiently means less money spent on hardware.  Theoretically anyway. SO WHAT IS REALLY COOL ABOUT CEP? CEP gives one the to submit queries like “select symbol, avg(shares) from trade_stream group by symbol over 5 minutes emit every 1 minute.”  The CEP engine would consume this query, and then start returning an average of sh... (more)

Will The Real Marketing Department Please Stand Up?

Well, it would appear that Michael Stonebraker may have hung up his research hat and joined the marketing team. First, read this, “Will the Real Column Stores Please Stand Up?” And now read my reply, which Vertica has yet to approve on their blog. Mike, You’re describing an implementation, not an algorithm.  An analogy would be saying that anything that didn’t look like Hadoop wasn’t map/reduce. Column stores exist because disk is slow – not because they’re some new and magical way to store data.  As a researcher, you’re well aware of the fact that it is impossible to prove that a... (more)

Building a Back Testing Platform for Algorithmic Trading

On this continuing series, I am examining thoughts and specific implementation details around building a back-testing platform for algo trading.  Eventually, we’ll see where complex event processing plays and how to implement it. Appendix to Part One – The Data Format Rather than looking at various database solutions first and then trying to define the problem in terms of those solutions, let’s first examine what market data looks like.  In its most simple form, market data looks like this (there’s usually a little more, but this is fine for our purposes): Date: The date of the ... (more)