Welcome!

Cloud Event Processing - Analyze, Sense, Respond

Colin Clark

Subscribe to Colin Clark: eMailAlertsEmail Alerts
Get Colin Clark via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Top Stories by Colin Clark

Every year, I like to decompress a bit and take a break. Usually, I like to go scuba diving – the dive sites I like are usually far removed from email, Twitter, Facebook, etc. and it gives me a chance to actually unplug, defrag, and think a bit. This year, the family went to Grand Cayman to experience some of the world’s best diving. Within Grand Cayman is Hell, a small township dedicated to tourism and aptly named given the attached photo. We visited Hell in between dives, and rather than make my ex-wife room reservations or send out postcards, I thought I’d amplify a few predictions I’d made earlier after the benefit of some quality thinking time. So here they are: Cloud based services will finally hit Wall Street in 2011. It makes a lot of sense for firms to offer shared access to things like market data and to provide elastic resource for its manipulation. It d... (more)

CEP in the Cloud – Current Status & Left To Do

Demo Overview The object of this demonstration is to show the classic Map/Reduce example of Word  Count in a CEP flavor which we’ll deploy into a cloud.  We’re almost there – all of the code should be wrapped up this week and we should be running in a cloud by the end of next week. RabbitMQ Currently, I have RabbitMQ running on a Mac Pro tower (Intel, of course).  It’s the latest version running on top of a locally compiled version of Erlang.  I’m using RabbitMQ as the bus for this project. Twitter OnRamp The Twitter OnRamp is up and running. The OnRamp’s job is to get data into... (more)

Cloud Aware or Cloud Built

So You’re Renting Xen Condo’s in the Cloud, Big Deal, What’s Your Next Act? Here’s another quote from the article, “21 Experts Define Cloud Computing” “The ‘cloud’ model initially has focused on making the hardware layer consumable as on-demand compute and storage capacity. This is an important first step, but for companies to harness the power of the cloud, complete application infrastructure needs to be easily configured, deployed, dynamically-scaled and managed in these virtualized hardware environments.” Kirill Sheynkman I like this quote too – but for different reasons th... (more)

Why CEP in the Cloud Makes Sense

CEP isn’t really about low latency.  The ability to do things quickly is important, just as in any system – especially those systems that grow and need to handle a lot of information.  Doing things quickly means doing things efficiently.  And doing things efficiently means less money spent on hardware.  Theoretically anyway. SO WHAT IS REALLY COOL ABOUT CEP? CEP gives one the to submit queries like “select symbol, avg(shares) from trade_stream group by symbol over 5 minutes emit every 1 minute.”  The CEP engine would consume this query, and then start returning an average of sh... (more)

Data Mining in Streaming Data

Lately, I’ve been working on some interesting projects involving not just the usual suspects of stream processing, but data mining within high velocity time series.  In conjunction with that effort, I’ve been doing a lot of research in the areas of symbolic representation, dimension reduction, clustering, indexing, classification, and anomaly detection.  A prolific  researcher in this area is Dr. Eamonn Keogh – I’ll be applying some of his team’s ideas so some interesting customer problems and telling you all about it here.  Let’s get started! TOO MUCH DATA In dealing with real... (more)