Swift: Lazily encapsulating chains of map, filter, flatMap

Question

I have a list of animals:

let animals = ["bear", "dog", "cat"]

And some ways to transform that list:

typealias Transform = (String) -> [String]

let containsA: Transform = { $0.contains("a") ? [$0] : [] }
let plural:    Transform = { [$0 + "s"] }
let double:    Transform = { [$0, $0] }

As a slight aside, these are analogous to filter (outputs 0 or 1 element), map (exactly 1 element) and flatmap (more than 1 element) respectively but defined in a uniform way so that they can be handled consistently.

I want to create a lazy iterator which applies an array of these transforms to the list of animals:

extension Array where Element == String {
  func transform(_ transforms: [Transform]) -> AnySequence<String> {

    return AnySequence<String> { () -> AnyIterator<String> in
      var iterator = self
        .lazy
        .flatMap(transforms[0])
        .flatMap(transforms[1])
        .flatMap(transforms[2])
        .makeIterator()

      return AnyIterator {
        return iterator.next()
      }
    }
  }
}

which means I can lazily do:

let transformed = animals.transform([containsA, plural, double])

and to check the result:

print(Array(transformed))

I'm pleased with how succinct this is but clearly:

        .flatMap(transforms[0])
        .flatMap(transforms[1])
        .flatMap(transforms[2])

is an issue as it means the transform function will only work with an array of 3 transforms.

Edit: I tried:

  var lazyCollection = self.lazy
  for transform in transforms {
    lazyCollection = lazyCollection.flatMap(transform) //Error
  }
  var iterator = lazyCollection.makeIterator()

but on the marked row I get error:

Cannot assign value of type 'LazyCollection< FlattenCollection< LazyMapCollection< Array< String>, [String]>>>' to type 'LazyCollection< Array< String>>'

which I understand because each time around the loop another flatmap is being added, so the type is changing.

How can I make the transform function work with an array of any number of transforms?

One WET solution for a limited number of transforms would be (but YUK!)

  switch transforms.count {
  case 1:
    var iterator = self
      .lazy
      .flatMap(transforms[0])
      .makeIterator()
    return AnyIterator {
      return iterator.next()
    }
  case 2:
    var iterator = self
      .lazy
      .flatMap(transforms[0])
      .flatMap(transforms[1])
      .makeIterator()
    return AnyIterator {
      return iterator.next()
    }
  case 3:
    var iterator = self
      .lazy
      .flatMap(transforms[0])
      .flatMap(transforms[1])
      .flatMap(transforms[2])
      .makeIterator()
    return AnyIterator {
      return iterator.next()
    }
  default:
    fatalError(" Too many transforms!")
  }

Whole code:

let animals = ["bear", "dog", "cat"]

typealias Transform = (String) -> [String]

let containsA: Transform = { $0.contains("a") ? [$0] : [] }
let plural:    Transform = { [$0 + "s"] }
let double:    Transform = { [$0, $0] }

extension Array where Element == String {
  func transform(_ transforms: [Transform]) -> AnySequence<String> {

    return AnySequence<String> { () -> AnyIterator<String> in
      var iterator = self
        .lazy
        .flatMap(transforms[0])
        .flatMap(transforms[1])
        .flatMap(transforms[2])
        .makeIterator()

      return AnyIterator {
        return iterator.next()
      }
    }
  }
}

let transformed = animals.transform([containsA, plural, double])

print(Array(transformed))

As you say I know the number of transforms as transforms.count but I can't see how to use that to make a loop of ".flatMap" operations because the type changes at each iteration. — Cortado-J, Jan 27 '19 at 08:40
i can see, the problem is this type as far as i know "iterator" can't take bounds to as it needs to implement directly the way you did, but some workaround can be applied by sending a flag to use another iterator or something. — Mohmmad S, Jan 27 '19 at 08:55
I'm wondering about changing the title of the question to: "Swift: Lazily encapsulating chains of map, filter, flatMap" as I think that would describe the essence of what this question is about much better but wondering if it's considered bad form to change the title of a question? — Cortado-J, Jan 27 '19 at 23:19

Martin R · Answer 1 · 2019-01-27T10:50:36.943

You can apply the transformations recursively if you define the method on the Sequence protocol (instead of Array). Also the constraint where Element == String is not needed if the transformations parameter is defined as an array of (Element) -> [Element].

extension Sequence {
    func transform(_ transforms: [(Element) -> [Element]]) -> AnySequence<Element> {
        if transforms.isEmpty {
            return AnySequence(self)
        } else {
            return lazy.flatMap(transforms[0]).transform(Array(transforms[1...]))
        }
    }
}

OOPer · Accepted Answer · 2019-01-27T22:27:14.987

Another approach to achieve what you want:

Edit: I tried:

var lazyCollection = self.lazy
for transform in transforms {
    lazyCollection = lazyCollection.flatMap(transform) //Error
}
var iterator = lazyCollection.makeIterator()

You were very near to your goal, if the both types in the Error line was assignable, your code would have worked.

A little modification:

var lazySequence = AnySequence(self.lazy)
for transform in transforms {
    lazySequence = AnySequence(lazySequence.flatMap(transform))
}
var iterator = lazySequence.makeIterator()

Or you can use reduce here:

var transformedSequence = transforms.reduce(AnySequence(self.lazy)) {sequence, transform in
    AnySequence(sequence.flatMap(transform))
}
var iterator = transformedSequence.makeIterator()

Whole code would be:

(EDIT Modified to include the suggestions from Martin R.)

let animals = ["bear", "dog", "cat"]

typealias Transform<Element> = (Element) -> [Element]

let containsA: Transform<String> = { $0.contains("a") ? [$0] : [] }
let plural:    Transform<String> = { [$0 + "s"] }
let double:    Transform<String> = { [$0, $0] }

extension Sequence {
    func transform(_ transforms: [Transform<Element>]) -> AnySequence<Element> {
        return transforms.reduce(AnySequence(self)) {sequence, transform in
            AnySequence(sequence.lazy.flatMap(transform))
        }
    }
}

let transformed = animals.transform([containsA, plural, double])

print(Array(transformed))

Very nice! – Minor remarks: The initial value in reduce can be `AnySequence(self)`, without the `lazy`. On the other hand, `AnySequence(sequence.lazy.flatMap(transform))` would avoid the creation of intermediate arrays. — Martin R, Jan 27 '19 at 12:33
Nice suggestion, frankly I had been forgotten that `flatMap` on `Sequence` generates an intermediate array. Thanks so much. — OOPer, Jan 27 '19 at 12:38
Thanks so much Martin and OOPer. Your combined thought have given me an elegant solution. Much appreciated. — Cortado-J, Jan 27 '19 at 15:21

Cristik · Answer 3 · 2019-01-28T17:36:54.783

How about fully taking this into the functional world? For example using (dynamic) chains of function calls, like filter(containsA) | map(plural) | flatMap(double).

With a little bit of reusable generic code we can achieve some nice stuff.

Let's start with promoting some sequence and lazy sequence operations to free functions:

func lazy<S: Sequence>(_ arr: S) -> LazySequence<S> {
    return arr.lazy
}

func filter<S: Sequence>(_ isIncluded: @escaping (S.Element) throws -> Bool) -> (S) throws -> [S.Element] {
    return { try $0.filter(isIncluded) }
}

func filter<L: LazySequenceProtocol>(_ isIncluded: @escaping (L.Elements.Element) -> Bool) -> (L) -> LazyFilterSequence<L.Elements> {
    return { $0.filter(isIncluded) }
}

func map<S: Sequence, T>(_ transform: @escaping (S.Element) throws -> T) -> (S) throws -> [T] {
    return { try $0.map(transform) }
}

func map<L: LazySequenceProtocol, T>(_ transform: @escaping (L.Elements.Element) -> T) -> (L) -> LazyMapSequence<L.Elements, T> {
    return { $0.map(transform) }
}

func flatMap<S: Sequence, T: Sequence>(_ transform: @escaping (S.Element) throws -> T) -> (S) throws -> [T.Element] {
    return { try $0.flatMap(transform) }
}

func flatMap<L: LazySequenceProtocol, S: Sequence>(_ transform: @escaping (L.Elements.Element) -> S) -> (L) -> LazySequence<FlattenSequence<LazyMapSequence<L.Elements, S>>> {
    return { $0.flatMap(transform) }
}

Note that the lazy sequences counterparts are more verbose that the regular Sequence ones, but this is due to the verbosity of LazySequenceProtocol methods.

With the above we can create generic functions that receive arrays and return arrays, and this type of functions are extremely fitted for pipelining, so let's define a pipeline operator:

func |<T, U>(_ arg: T, _ f: (T) -> U) -> U {
    return f(arg)
}

Now all we need is to feed something to these functions, but to achieve this we'll need a little bit of tweaking over the Transform type:

typealias Transform<T, U> = (T) -> U

let containsA: Transform<String, Bool> = { $0.contains("a") }
let plural:    Transform<String, String> = { $0 + "s" }
let double:    Transform<String, [String]> = { [$0, $0] }

With all the above in place, things get easy and clear:

let animals = ["bear", "dog", "cat"]
let newAnimals = lazy(animals) | filter(containsA) | map(plural) | flatMap(double)
print(Array(newAnimals)) // ["bears", "bears", "cats", "cats"]

Wow! Very nice @Cristik. When I first started thinking about this I did wonder if there might be a way to create an operator to combine filter, map, flatMap but have only been exploring functional thinking for a couple of months so my brain couldn't fathom it - so great to see your methods. One thing that isn't clear to me... how would I access the result lazily? let operation = filter(containsA) | map(plural) | flatMap(double) for x in operation(animals) { print(x) } is not lazy? — Cortado-J, Jan 27 '19 at 22:08
@Adahus good question, the laziness in this approach doesn't seem to function that well so I updated the answer and removed the `lazy` calls since they seem to help with almost nothing. I'll try and get back with a really "lazy" solution :) — Cristik, Jan 27 '19 at 22:22
I'm wondering if the free functions need to be passing AnySequence? — Cortado-J, Jan 27 '19 at 23:00
@Adahus actually, I was able to solve the problem via protocols only, I updated the answer and now if you provide a lazy sequence as a starting point then the whole pipeline will be lazy. — Cristik, Jan 28 '19 at 17:37
Wow again! This is awesome. Thanks for putting in the time to write this. Even though this isn't many lines of code it represents quite a significant set of methods. My next challenge is to ponder how to allow a lazy cartesian product to be added to the pipeline... — Cortado-J, Jan 28 '19 at 19:38
Just discovered https://github.com/pointfreeco/swift-overture which is similar territory. (I have no connection with Pointfree.) — Cortado-J, Jan 30 '19 at 22:29

Swift: Lazily encapsulating chains of map, filter, flatMap

3 Answers3

Linked