7

Is it possible to grab the AST of a block from Ruby itself?

I've had a look at both ParseTree and ruby_parser, but they both seem to have sketchy support (from what I've read) for Ruby 1.9.2. I need something that works well with 1.9.2.

1 Answers1

6

Ripper is included in MRI 1.9 out of the box.

ruby-1.9.2-p180 :004 > require 'ripper'
 => true
ruby-1.9.2-p180 :005 > Ripper.sexp("def a; end")
 => [:program, [[:def, [:@ident, "a", [1, 4]], [:params, nil, nil, nil, nil, nil], [:bodystmt, [[:void_stmt]], nil, nil, nil]]]]

In 1.8, Ruby executes the code by traversing the AST, so it is possible to get the AST for a given method/block. In 1.9 it is not so; the code is first parsed, then converted to YARV bytecode, and then executed. Not the source, nor AST is kept after the translating step, and the latter is not reversible; hence you cannot get the AST for a block in 1.9.

Catherine
  • 22,492
  • 3
  • 32
  • 47
  • How would I use Ripper to grab the AST from a block? I'm assuming I'd have to get the source code for the block somehow. –  Aug 01 '11 at 06:28
  • Thanks for the info! It seems like you need to do `require 'ripper'` for it to work. – David Grayson Aug 01 '11 at 06:28
  • Oh. Sorry, I should have read your post more carefully. I've updated the answer. – Catherine Aug 01 '11 at 06:34
  • 3
    @charlie you can use sourcify gem to get the source from the block – Surya Aug 22 '11 at 16:09
  • It's incorrect that you can't get the source of a block in Ruby 1.9+ you can't do it with ruby but e.g. sourcify works just well – Rune FS Mar 14 '13 at 08:19
  • @RuneFS, nope; sourcify uses a fundamentally imperfect approach and cannot process all code. – Catherine Mar 14 '13 at 19:23
  • @whitequark yes that gem can handle all code, yes it is doable to create a gem that can and so far I've been able to get all my code back (with a few work arounds because yes it's in complete which the version number strongly indicates) – Rune FS Mar 15 '13 at 07:42
  • @RuneFS, `lambda { lambda { 1 } }.call.to_source` – Catherine Mar 15 '13 at 17:34
  • @whitequark I'm not quite sure what your comment is supposed to suggest. .to_source produces source code not AST. that would be to_sexp. calling to_sexp as you do wouldn't work (as we both agree there are cases sourcify can't handle) however in this case just insert a newline after the first `lambda {` at it will work. So really not sure what you were suggesting, sorry – Rune FS Mar 18 '13 at 11:48
  • @RuneFS I have demonstrated that `sourcify` cannot parse all unmodified Ruby code, contrary to what you say. As a practical matter, such code might be autogenerated. – Catherine Mar 18 '13 at 14:16
  • I never postulated it could (yes there's a typo "that gem _can't_ .." however that should be obvious from the rest of that comment, where I both say that reworks might be required and that the gem is incomplete. However that does not affect my point. You claim it can't be done (not that sourcify can't but that it in general can't) and I claim you can reverse engineer byte code to semantically equivalent source code (as with all other languages/VMs) – Rune FS Mar 19 '13 at 08:59
  • @RuneFS you can trivially parse the source, but unless ruby reports source location with enough resolution to distinguish lambdas located on the same line, there is no point is doing that. And MRI's parser is very unlikely to ever report column numbers. – Catherine Mar 20 '13 at 10:12