6

Current status of truffleruby says:

TruffleRuby is progressing fast but is currently probably not ready for you to try running your full Ruby application on. Support for critical C extensions such as OpenSSL and Nokogiri is missing.

Why does truffleruby need C extensions? It's built on GraalVM which is built on top of the JVM, it is in fact a fork of JRuby:

TruffleRuby is a fork of JRuby, combining it with code from the Rubinius project, and also containing code from the standard implementation of Ruby, MRI.

Can't they use JRuby world gems instead of depending on their C variants?

EDIT link to the issue on github

bbozo
  • 7,075
  • 3
  • 30
  • 56

2 Answers2

14

Running C extensions is hard because the C extension API is just the entire internals of MRI exposed as a header file. You aren't programming against a clean API - you're programming against all the implementation details and internal design decisions of MRI.

JRuby's Java extensions have exactly the same problem! The JRuby Java extension API is just the entire internals of JRuby, and you aren't programming against an API, instead you're programming against all the implementations details and design decisions of JRuby.

We plan to eventually tackle both problems in the same way - which is to add another level of abstraction over the C or Java code using an interpreter which we can intercept and redirect when required, so that it believes it is running against MRI or JRuby internals, but really we redirect these to our internals.

We think C extensions are more important, so we're tackling those first. We haven't really started on Java extensions yet, but we have started the underlying interpreter for Java that we'll use.

This video explains all

https://youtu.be/YLtjkP9bD_U?t=1562

Chris Seaton
  • 290
  • 2
  • 7
  • That surprises me because it's dead simple to call Java code from JRuby and much more complicated to run C code from Ruby, I had an impression that JRuby integrates with their extensions much much more cleanly, I guess the other direction is more tricky? – bbozo Jan 22 '17 at 17:23
  • Also there's the userbase issue, people keep away from JRuby due to unfamiliarity with the JVM and high memory requirements being an issue on cheap VMs on Heroku and EC2 - I guess TruffleRuby shares those issues thanks to the JVM? On the other hand, one thing that pulls people *to* JRuby is access to the Java eco system, people at Stripe use it due to EmvJ, we used it only when a Mastercard project forced us to implement really robust XML signing which simply didn't exist in the MRI world & when we needed to deploy to AS400. – bbozo Jan 22 '17 at 17:29
  • That's why I'm wondering, you're creating a super awesome fast ruby implementation, but one that requires you to get comfortable with JVM management, debugging and memory use, but at the same time, at least for the forseeable future, scraps the benefits of the Java ecosystem access and super-portability, that's what's bugging me. Also I thought since you guys forked JRuby it would be super-simple**r** to go down that road. – bbozo Jan 22 '17 at 17:33
  • 1
    You can call Java code from Ruby and Ruby code from Java just fine with Truffleized JRuby, AFAIK. It is JRuby extensions that don't work. In the video Chris linked, he explains that the JRuby extension interface has the same problem as the YARV extension interface: there isn't one. He also explains that they have already solved the "hard" problem with C extensions, and that the solution for the JRuby extensions is basically the same: write a JVM bytecode interpreter in Truffle. So, as I understand it, they chose to do C first, because it is at the same time harder and more beneficial. – Jörg W Mittag Jan 22 '17 at 17:37
  • @JörgWMittag maybe it's my lack of understanding, but what is a JRuby extension but a piece of Java code executed from Ruby? If they support running Java code from Ruby that would basically mean they support all Java extensions people plugged into their gems, no? – bbozo Jan 22 '17 at 17:40
  • @JörgWMittag an even more informative talk https://youtu.be/b1NTaVQPt1E?t=3363 <3 I guess the bottom line is that they want to compile C & Ruby into a common AST and then push it to an optimizer. They don't intend to just call Java methods - they intend to optimize them together after reducing them to a common denominator and just calling Java would leave them with a bunch of regressions they're out to avoid in the first place. – bbozo Jan 22 '17 at 19:01
  • Open question is why don't they use Java interoperabilty to gain access to the same market where JRuby has access to and take time to optimize other stuff case-by-case. I guess they're not interested in rushing it to production just now, which is very reasonable on their part. Kudos again – bbozo Jan 22 '17 at 19:03
  • @bbozo yes that second video link explains in more technical depth why we had to move away from the internal design of JRuby to get the speed we wanted. That means that the classes that the Java extensions hook into simply aren't there any more. – Chris Seaton Jan 22 '17 at 20:17
3

You have already gotten a good answer by the project lead himself, but I want to offer a different point of view:

Why does truffleruby need C extensions?

It doesn't need them. But they do exist and there is code out there which uses them, and it would sure be nice to be able to run that code.

Jörg W Mittag
  • 363,080
  • 75
  • 446
  • 653
  • Yeah, but it's interesting they chose to go for C extensions first, and Java extensions later considering the code runs in the JVM, no? :) – bbozo Jan 22 '17 at 17:30
  • Well, there are lots of C extensions that people rely on. There aren't many JRuby extensions that have no C or Ruby equivalent. The converse is not true. – Jörg W Mittag Jan 22 '17 at 17:33