3

I am considering using StreamInsight for an CEP system I'm working on, however the scale out story doesn't seem to exist in any of the documentation I could find online. StreamInsight seems to be designed to run on a single server. The size of the system I'm looking at could reach tens of thousands of transactions per second and needs to have a high SLA. So I need to understand how (if anyone has) scaled StreamInsight deployments horizontally, meaning over multiple servers to achieve high availability.

It looks like the SI team was thinking of this in 2011, but its not clear anything has come from that since the latest build of SI is from 2012.

Paul Fryer
  • 9,268
  • 14
  • 61
  • 93
  • I did find one model online that suggests using SI in the reduce portion of map/reduce with Hadoop. http://blogs.msdn.com/b/microsoft_business_intelligence1/archive/2012/02/22/big-data-hadoop-and-streaminsight.aspx – Paul Fryer May 30 '14 at 17:29

1 Answers1

4

There is no official "scale out" story for StreamInsight. However, one solution would be to have multiple servers running the queries and then, based on a parameter, only a single server actually "outputting" the results. It would be pretty complex but certainly doable. For resiliency, you can use checkpointing ... this will write the state of the event engine to disk so that you can recover the running queries. Depending on the queries that you have, this may or may not be helpful. One comment - "tens of thousands" of events/second is something that can easily be done on a single server.

DevBiker
  • 451
  • 2
  • 4