Chaining enumerators that yield multiple arguments

Question

I'm trying to figure out how Ruby handles chaining enumerators that yield multiple arguments. Take a look at this snippet:

a = ['a', 'b', 'c']

a.each_with_index.select{|pr| p pr}
# prints:
# ["a", 0]
# ["b", 1]
# ["c", 2]

a.each_with_index.map{|pr| p pr}
# prints:
# "a"
# "b"
# "c"

Why does select yield the arguments as an array, whereas map yields them as two separate arguments?

Very interesting. It's a very rarely seen high quality Ruby question asked by a newcomer recently. You are a wise man. — sawa, Jan 24 '13 at 02:49
I've sent mail about this issue to ruby-core, and I'll update here when I hear back. [link](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/51614) — Ben Weissmann, Jan 24 '13 at 04:39
The best is to sign up to http://bugs.ruby-lang.org/issues/, I also posted this as an answer. — Boris Stitnicky, Jan 24 '13 at 04:40

ian · Answer 1 · 2013-01-24T03:44:06.930

5

Try:

a.each_with_index.map{|pr,last| p "pr: #{pr} last: #{last}"}

map is automatically deconstructing the values passed to it. The next question is why is it doing this deconstruction and select isn't?

If you look at the source given on the Rdoc page for Array they're virtually identical, select only differs in that it does a test on the value yielded. There must be something happening elsewhere.

If we look at the Rubinius source (mainly because I'm better with Ruby than C;) for map (aliased from collect) it shows us:

each do |*o|

so it's splatting the arguments on the way through, whereas select (aliased from find_all) does not:

each do

again, the design decision as to why is beyond me. You'll have to find out who wrote it, maybe ask Matz :)

I should add, looking at the Rubinius source again, map actual splats on each and on yield, I don't understand why you'd do both when only the yield splat is needed:

  each do |*o|
    ary << yield(*o)
  end

whereas select doesn't.

each do
  o = Rubinius.single_block_arg
  ary << o if yield(o)
end

edited Jan 24 '13 at 03:44

answered Jan 24 '13 at 03:17

ian

12,003
9
51
107

That's what i figured. I'm still interested in the _why_ -- and whether there's any documentation on which methods use which technique. Any idea what the best way to get in touch with Matz/the core ruby devs is? – Ben Weissmann Jan 24 '13 at 04:30
I will post this as a regular answer. – Boris Stitnicky Jan 24 '13 at 04:33
@BenWeissmann then the answer Boris gave is most helpful. I'd suggest the way right now to find out which methods use which technique is either to try a desconstruction using a pattern in IRB (e.g. `do |(a,b)|`…) or look at the Rubinius source (as it's supposed to match the MRI implementation) and see if `yield` takes a splat. The link I gave above will take you to the right place, any of kernel/common/xxx will be the core. You could always ask the Rubinius contributors in #rubinius on Freenode, since they reimplemented it. – ian Jan 24 '13 at 12:45

score 4 · Answer 2 · answered Jan 24 '13 at 03:24

According to the MRI source, it seems like the iterator used in select splats its arguments coming in, but map does not and passes them unpacked; the block in your latter case silently ignores the other arguments.

The iterator used in select:

static VALUE
find_all_i(VALUE i, VALUE ary, int argc, VALUE *argv)
{
    ENUM_WANT_SVALUE();

    if (RTEST(rb_yield(i))) {
        rb_ary_push(ary, i);
    }
    return Qnil;
}

The iterator used in map:

static VALUE
collect_i(VALUE i, VALUE ary, int argc, VALUE *argv)
{
    rb_ary_push(ary, enum_yield(argc, argv));

    return Qnil;
}

I'm pretty sure the ENUM_WANT_SVALUE() macro is used to turn the value passed into the block into a splat array value (as opposed to a tuple with the latter arguments silently ignored). That said, I don't know why it was designed this way.

score 3 · Answer 3 · answered Jan 24 '13 at 03:36

Let's see MRI source in enum.c. As @PlatinumAzure said, the magic happens in ENUM_WANT_SVALUE():

static VALUE
find_all_i(VALUE i, VALUE ary, int argc, VALUE *argv)
{
    ENUM_WANT_SVALUE();

    if (RTEST(rb_yield(i))) {
        rb_ary_push(ary, i);
    }
    return Qnil;
}

And we can find this macro actually is: do {i = rb_enum_values_pack(argc, argv);}while(0).

So Let's continue dive into rb_enum_values_pack function:

VALUE
rb_enum_values_pack(int argc, VALUE *argv)
{
    if (argc == 0) return Qnil;
    if (argc == 1) return argv[0];
    return rb_ary_new4(argc, argv);
}

See? The arguments are packed by rb_ary_new4, which is defined in array.c.

score 3 · Accepted Answer · answered Jan 24 '13 at 04:36

From the discourse so far, it follows that we can analyze the source code, but we do not know the whys. Ruby core team is relatively very responsive. I recommend you to sign in at http://bugs.ruby-lang.org/issues/ and post a bug report there. They will surely look at this issue at most within a few weeks, and you can probably expect it corrected in the next minor version of Ruby. (That is, unless there is a design rationale unknown to us to keep things as they are.)

Chaining enumerators that yield multiple arguments

4 Answers4