If each is an Enumerable and with_index is an Enumerable, then why can each iterate arrays but not with_index?

Question

I think the question is self-explanatory: If each is an Enumerable and with_index is an Enumerable, then why can each iterate arrays but not with_index?

I'll provide an example:

 alphabet =  [:a, :b, :c]
 => [:a, :b, :c] 
alphabet.each.class
 => Enumerator 
alphabet.each.with_index.class
 => Enumerator 
alphabet.each
 => #<Enumerator: [:a, :b, :c]:each> 
2.1.2 :036 > alphabet.with_index
NoMethodError: undefined method `with_index' for [:a, :b, :c]:Array

It seems unintuitive to me that both are Enumerables, yet each can respond_to Array but with_index cannot. Because of this, I get forgetful when I try to build these Enumerables:

alphabet.each.with_index(1).reduce({}) do |acc, (letter,i)|
end

Sometimes I make this mistake:

alphabet.with_index(1).reduce({}) do |acc, (letter,i)|
end

forgetting about the dependency that with_index must be chained to each.

Answer 1

each is not an Enumerable, it returns an Enumerator (pay attention to the spell) which is_a?(Enumerable).

Why [1,2,3].with_index doesn't work? Because with_index is an instance method of Enumerator, not a method of Enumerable.

Answer 2

Enumerable module

All methods in the module Enumerable are instance methods that require their receiver to be an enumerator (an instance of the class Enumerator). All classes that include the module Enumerable (using Module#include) must possess an instance method each, which returns an enumerator. Three examples for built-in classes are Arrays#each, Hash#each and Range#each.¹ To include Enumerable in a custom class, therefore, one must define a method each (that returns an enumerator) on that class.

When a method contained in Enumerable is executed on an instance of a class that includes Enumerable, Ruby inserts the method each between the instance and the Enumerable method.

For example, you can think of [1,2,3].map { |n| 2*n } #=> [2,4,6] as [1,2,3].each.map { |n| 2*n } (using Array#each), { :a = 1, :b => 2 }.map { |k,v| v } #=> [1,2] as { :a = 1, :b => 2 }.each.map { |k,v| v } (using Hash#each) and (1..4}.map { |n| 2*n } #=> [2,4,6] as (1..4}.each.map { |n| 2*n } (using Range#each).

with_index

Now let's consider the method with_index. Which modules (including classes, of course) have an instance method with_index?

ObjectSpace.each_object(Module).select { |m| m.instance_methods.include?(:with_index) }
  #=> [Enumerator::Lazy, Enumerator]

(where Enumerator::Lazy.superclass #=> Enumerator) See ObjectSpace#each_object.

It follows that invoking with_index on any class other than these two will raise a no-method exception. That includes, for example, [1,2,3].with_index and { :a=>1 }.with_index. However, classes that have a method each that returns an enumerator can make use of the method Enumerator#with_index by (explicitly) inserting each between an instance of the class and with_index.

For example²,

enum = [1,2,3].each.with_index 
  #=> #<Enumerator: #<Enumerator: [1, 2, 3]:each>:with_index>

and then perhaps used thusly:

enum.to_a
  #=> [[1, 0], [2, 1], [3, 2]]

This works because

[1,2,3].each.class
  #=> Enumerator

so all instance methods of the class Enumerator (including with_index) can be invoked on the enumerator [1,2,3].each.

^{1 A complete list is given by ObjectSpace.each_object(Class).select {|k| k.instance_methods.include?(:each)}. 2 One would generally write enum = [1,2,3].each_with_index (see Enumerable#each_with_index) rather than enum = [1,2,3].each.with_index, but the latter is sometime used to make use of the fact that with_index takes an argument (default 0) that specifies the index base. For example, [1,2,3].each.with_index(1).to_a #=> [[1, 1], [2, 2], [3, 3]].}