9

What are advantage/disadvantage of using akka stream vs spark stream for stream processing? like, built in back pressure, performance, fault tolerance, built in transformation, flexibility etc. I'm NOT asking akka vs spark pros/cons strictly streaming component. Also I'm NOT asking under the hood framework architecture difference.

Hiren
  • 708
  • 1
  • 7
  • 14
  • Pretty sure Spark is built on Akka so you are really just getting more concise syntax, more user-friendliness, built in fault tolerance (supervisors and what not), and a few more features with Spark. If you have a more custom need such as running a ton of varying jobs with different actors/flows on a single cluster, you should consider Akka over Spark. Otherwise, more features is probably a good thing. That is a more general answer. – Andrew Scott Evans May 16 '17 at 16:25
  • 3
    I believe Spark moved away from Akka after 1.5. https://issues.apache.org/jira/browse/SPARK-6602 – Achilleus Nov 24 '17 at 08:16

1 Answers1

21

Akka Streams and Spark streams are from 2 different lands. Do not let the word "streams" confuse you.

Akka streams implement something called reactive manifesto which is great to achieve really low latency and provide a lot of operators to write declaratively transformations over streams easily. More about this on https://doc.akka.io/docs/akka/2.5.4/scala/stream/stream-introduction.html#motivation.

Spark Streaming aka Structured Streaming as of 2.2 is still a micro-batch approach to process a huge amount of data(Big Data).Events are collected and then processed periodically in small batches every few seconds.

Akka streams is basically not a distributed and do not scale out across clusters, unlike Spark.Akka streams use actor model of Akka to achieve Concurrency.

Akka streams is a toolkit and Spark is a framework. PS: Even I had the same question couple of months back. Took a while to get my answers. Hope its helpful.

Achilleus
  • 1,824
  • 3
  • 20
  • 40
  • One thing that doesn't make sense, why akka wouldn't scale, sharding wont work here? – tesnik03 Mar 07 '19 at 17:30
  • I don’t know if there is a concept of sharding in akka but there is akka remoting a part of akka project which can be used to spin up akka actors across machine. – Achilleus Mar 07 '19 at 17:32
  • Akka definitely provides a rich set of tools to build distributed systems, that's what akka cluster is built for, see https://doc.akka.io/docs/akka/current/typed/cluster.html to learn more. – yiksanchan Aug 26 '20 at 00:16