3

My understanding is that IO Implementation like cats-effetcs or Zio use fibers, and this on the JVM.

I wonder what's the underlying lib or framework they are using, if the JVM e.g. 11 does not officially support fibers yet ?

MaatDeamon
  • 9,532
  • 9
  • 60
  • 127
  • 3
    First, there is no single JVM version yet that has fibers built-in. - Second, AFAIK _(if wrong I would appreciate being corrected)_, there is no a fiber building framework out there intended to be used by libraries to create their own low-level fibers. - Third, all those libraries implement their own fibers from scratch. - Fourth, at least for **CE3** they are not interested in project loom since it should not improve _(rather it seems it would even decrease)_ the performance of their own scheduler; but time will tell what will happen. – Luis Miguel Mejía Suárez Apr 21 '21 at 13:34
  • Fifth, check [this](https://systemfw.org/talks.html#scala-world-2019) & [this](https://typelevel.org/blog/2021/02/21/fibers-fast-mkay.html). – Luis Miguel Mejía Suárez Apr 21 '21 at 13:35
  • Thanks for the input on this – MaatDeamon Apr 21 '21 at 13:35
  • @LuisMiguelMejíaSuárez I do not know much scala, but that "since it should not improve" is a very bold claim, considering that the plan is to move _everything_ to fibers under the hood. – Eugene Apr 21 '21 at 15:14
  • 1
    @Eugene so I am somewhat quoting what I have heard from the authors of **CE3** so I am not strong about defending that statement; moreover, that is why I explicitly said only time will tell. - Second, being honest I am far from being an expert in the internals of **CE3** or project Loom so I really can not properly explain why that will be the case. - However, what I understood is that right **CE3** already provides fibers and a very performant scheduler that takes advantage of some contextual information of the computation in order to properly schedule it... – Luis Miguel Mejía Suárez Apr 21 '21 at 15:20
  • ... that scheduler was based in the **tokio** project from **Rust**. And what I understood is that Loom fibers would not help in this case, because they are actually more opaque, they lack that contextual information and would not be properly scheduled. - So again, we will need to wait to see how those two things will interact. IIRC, the claim was made in [this videos](https://www.youtube.com/watch?v=JrpFFRdf7Q8&ab_channel=Konfy) maybe I just misunderstood the author, if you check it out and find a mistake in my quote, please let me know :) – Luis Miguel Mejía Suárez Apr 21 '21 at 15:22
  • @LuisMiguelMejíaSuárez understandable. I do not have the needed knowledge to comment or make a reasonable argument. I just found that a bit weird, thank you for the follow up. – Eugene Apr 21 '21 at 15:23
  • Thanks for the your exchange guys it helps. @LuisMiguelMejíaSuárez those links and mentioning Tokio is good. To some degree the motivation behind the question is somehow being able to have a sense of how things work under the hood without necessary fully understanding the detail. This is important I guess for folks who would want to introduce cats or zio for doing concurrent work. People trust Java and the executor context and what not. If you start saying I am using things that is not officially Java supported to your hierarchy especially for things like concurrency mechanism ... – MaatDeamon Apr 21 '21 at 16:13
  • One better own it and explain when there are bugs what is going on and how to fix it. Executor context and their scheduler are battle tested older stuff and there is Java support if there is issue and what not. Theses part of the code are rather critical. Cats and zio are open source so no support, u r on your own if u introduce it to do things that involve heavy concurrency .... so having a sense of the underlying mechanism, how battle tested the thing is, how fast issue are fixed and what not is important ... – MaatDeamon Apr 21 '21 at 16:16
  • If however they were using officially supported Java mechanism for these “critical” part that makes it more easy in one organization. I suspect bugs at the higher layer of the library are easier to detect or manage, but scheduler of concurrency stuff that’s a different story – MaatDeamon Apr 21 '21 at 16:19
  • @MaatDaemon I am pretty sure I have found more bugs and worse support with Java official things than with **cats-effect**, just my two cents. - Also, note that most of those things are using things from the standard JDK under the hood anyways like at the end things are running in a `Thread` and things like `Ref` are wrappers over `AtomicReference`. – Luis Miguel Mejía Suárez Apr 21 '21 at 16:22
  • Ahaha got ya! But corporate is about what the big bosses heard is to be trusted! Unfortunately I am in that world !!! Anyway point taken – MaatDeamon Apr 21 '21 at 16:24
  • 1
    Both `zio` and `cats-effect` are very high quality projects, and pretty widely used in the Scala industry. I wouldn't think they're more of a "bug risk" than the official Java implementation. Even if they were, you have to factor in the amount of bugs that expressing your concurrency patterns in a sane way is going to avoid! ZIO and CE3 give you that, Java primitives do not. – francoisr May 01 '21 at 07:20

1 Answers1

3

The JVM has no official support for fibers. This is however in the works with Project Loom.

In the meantime, effect libraries like ZIO and Cats Effect are maintaining a pool of thread and rolling their own thread scheduling implementation, as stated in the question comments by Luis.

If the fact that fiber implementations are library-specific is making you think that they're a bug-risk in open-source libraries, please keep in mind the following: fibers are a low-level concept in both ZIO and Cats Effect. The point of these libraries is to largely decouple what is run from the details of exactly how it runs. They still offer flexibility for you to express how to run things, but this part of the API is not used in 90% of cases. What ZIO and Cats Effect bring you is a way to express concurrent tasks with nice type-safe primitive. This is exactly what will help you avoid bugs.

francoisr
  • 4,407
  • 1
  • 28
  • 48
  • I am sold btw. The question was not so much about how more effective it is to do concurrent work in a type-safe way, but rather managing the humans around it especially hierarchy in corporate. I needed to have enough info to defend the case. To give an example, when akka and ultimately akka stream got rolled out, they did way more work to explain their inner working, and you have a company that backs it and support it. That is just by essence easier to sell. – MaatDeamon May 01 '21 at 07:45
  • However I agree with all your points and pointers and it helps. I feel more at ease with explaining my choices and what not. – MaatDeamon May 01 '21 at 07:46
  • Just another stupid illustration, my company is forcing one of our unit that developed a project prior to when they were acquire, that is using SOLR, to ElasticSearch, just for those reason, nothing to do with technicality. So ultimately it is all about making a proper case, and that require knowing what you are talking about end to end, which was the original motivation behind my question – MaatDeamon May 01 '21 at 07:52
  • 2
    I suggest you watch a few "FP sales pitch" video from some recent FP conferences, and take a few concrete examples of how using effect systems ultimately makes life easier, even though there is somewhat of a learning curve. Much like FP is now in every mainstream language in the form of closures/lambdas, I believe you will see this trend spread around in the next few years. – francoisr May 01 '21 at 08:18