8

Let's say I have two enumerators, enum1 and enum2 that must be lazily iterated through (because they have side effects). How do I construct a third enumerator enum3 where enum3.each{|x| x} would lazily return the equivalent of enum1 + enum2?

In my real world use case, I'm streaming in two files, and need to stream out the concatenation.

Alex Altair
  • 3,246
  • 3
  • 21
  • 37

3 Answers3

10

This seems to work just how I want;

enums.lazy.flat_map{|enum| enum.lazy }

Here's the demonstration. Define these yielding methods with side-effects;

def test_enum
  return enum_for __method__ unless block_given?
  puts 'hi'
  yield 1
  puts 'hi again'
  yield 2
end  

def test_enum2
  return enum_for __method__ unless block_given?
  puts :a
  yield :a
  puts :b
  yield :b
end  

concated_enum = [test_enum, test_enum2].lazy.flat_map{|en| en.lazy }

Then call next on the result, showing that the side effects happen lazily;

[5] pry(main)> concated_enum.next
hi
=> 1
[6] pry(main)> concated_enum.next
hi again
=> 2
Alex Altair
  • 3,246
  • 3
  • 21
  • 37
  • 1
    Thanks so much for the `flat_map` solution. Even though we'd already verified the laziness of `flat_map`, this didn't occur to us, to use it as a lazy append! :) – ms-ati Aug 29 '16 at 20:13
  • The drawback of this solution is that `concated_enum.size` always returns `nil`. – skalee Mar 08 '18 at 13:00
  • @skalee I think that's a drawback of lazy enums; you can never know how big they are until you've iterated through them. Their size might be a function of how long you take to iterate. They can even be infinite. – Alex Altair Mar 08 '18 at 18:42
  • 1
    `enums.lazy.flat_map(&:lazy)` is a bit cleaner. might also add an explanation of how it works. – Jonah Jan 05 '19 at 00:07
  • More recently, since Ruby 2.6 you can use [Enumerable#chain](https://ruby-doc.org/core-2.6/Enumerable.html#method-i-chain)/[Enumerator::Chain](https://ruby-doc.org/core-2.6/Enumerator/Chain.html). See my answer below. – bts Nov 07 '20 at 08:15
1

Here's some code I wrote for fun awhile back with lazy enumeration thrown in:

def cat(*args)
  args = args.to_enum

  Enumerator.new do |yielder|
    enum = args.next.lazy

    loop do
      begin
        yielder << enum.next
      rescue StopIteration
        enum = args.next.lazy
      end
    end
  end
end

You would use it like this:

enum1 = [1,2,3]
enum2 = [4,5,6]
enum3 = cat(enum1, enum2)

enum3.each do |n|
  puts n
end
# => 1
#    2
#    3
#    4
#    5
#    6

...or just:

cat([1,2,3],[4,5,6]).each {|n| puts n }
Jordan Running
  • 102,619
  • 17
  • 182
  • 182
1

Since Ruby 2.6 you can use Enumerable#chain/Enumerator::Chain:

a = [1, 2, 3].lazy
b = [4, 5, 6].lazy

a.chain(b).to_a
# => [1, 2, 3, 4, 5, 6]

Enumerator::Chain.new(a, b).to_a
# => [1, 2, 3, 4, 5, 6]
bts
  • 328
  • 1
  • 4
  • 11