Colin Clark

In this video I show how to use DarkStar to filter the Twitter stream ... (more)

Building a Back Testing Platform for Algorithmic Trading

On this continuing series, I am examining thoughts and specific implementation details around building a back-testing platform for algo trading.  Eventually, we’ll see where complex event processing plays and how to implement it. Appendix to Part One – The Data Format Rather than looking at various database solutions first and then trying to define the problem in terms of those solutions, let’s first examine what market data looks like.  In its most simple form, market data looks like this (there’s usually a little more, but this is fine for our purposes): Date: The date of the ... (more)

CEP & RuleBots & OnRamps – Oh My!

I like Streambase’s announcement regarding their open source repository for CEP related stuff. And imitation is the sincerest form of flattery, so I thought I’d follow suit. In my glossary of terms, I deal with RuleBots and OnRamps. RuleBots RuleBots are pieces of code that subscribe to streams, do something of interest, and then put a result back on a stream. We’re releasing RuleBots with logic pertaining to Regulation NMS surveillance rules in the areas of: Trade Through, Unusual Activity, NBBO Calculator, Late Trading, Front Running, Wash Sales, Short Sales, Locked/Crossed, a... (more)

It’s Time to Kill the Elephant

Google started using MapReduce about 10 years ago.  Somewhere between there and now, Doug Cutting decided that he could copy it while at Yahoo and Hadoop was born.  Doug now works at a company named Cloudera who bills themselves as providing the “only solution that manages Apache Hadoop across the enterprise.”  Hadoop has been around for so long that even leading analyst firms are covering it, claiming that if your organization is an early adopter, you need to be looking at Hadoop.  Hear that Luddites?  Time to get moving. Hadoop Is Picking Up Speed MAYBE THERE’S A REASON FOR T... (more)

Cloud vs Grid

IBM Session at Cloud Expo I’ve been getting a lot of email, and there have been some posts on this blog, regarding CEP based services in the cloud. So as we go down this road, I thought it would behoove us to examine what a cloud is and isn’t and what a grid is and isn’t. I found this introductory article from IBM.  It’s a good start.  I’ll be looking for some more articles – if you’ve got anything you’d like to add to this conversation, please feel free to contact me or add your comments below. Personally, my experience with grid comes from Capital Markets and calculating pric... (more)