14

I have been looking into FP languages (off and on) for some time and have played with Scala, Haskell, F#, and some others. I like what I see and understand some of the fundamental concepts of FP (with absolutely no background in Category Theory - so don't talk Math, please).

So, given a type M[A] we have map which takes a function A=>B and returns a M[B]. But we also have flatMap which takes a function A=>M[B] and returns a M[B]. We also have flatten which takes a M[M[A]] and returns a M[A].

In addition, many of the sources I have read describe flatMap as map followed by flatten.

So, given that flatMap seems to be equivalent to flatten compose map, what is its purpose? Please don't say it is to support 'for comprehensions' as this question really isn't Scala-specific. And I am less concerned with the syntactic sugar than I am in the concept behind it. The same question arises with Haskell's bind operator (>>=). I believe they both are related to some Category Theory concept but I don't speak that language.

I have watched Brian Beckman's great video Don't Fear the Monad more than once and I think I see that flatMap is the monadic composition operator but I have never really seen it used the way he describes this operator. Does it perform this function? If so, how do I map that concept to flatMap?

BTW, I had a long writeup on this question with lots of listings showing experiments I ran trying to get to the bottom of the meaning of flatMap and then ran into this question which answered some of my questions. Sometimes I hate Scala implicits. They can really muddy the waters. :)

Community
  • 1
  • 1
melston
  • 2,198
  • 22
  • 39
  • Well isn't `xss.flatten` just shorthand for `xss.flatMap(identity)`, so `flatten` derives from `flatMap`, rather than the other way round? Though for explaining things, I suppose `flatten` is a bit easier to get your head around. – Luigi Plinge Dec 12 '15 at 01:20
  • @LuigiPlinge, yeah I have seen that explained as well. I know many of the manipulations available with `flatMap` and how it can be used. I just don't think I really *understand* it. There seems to be a concept that I just can't quite grasp somewhere. – melston Dec 12 '15 at 03:26

5 Answers5

35

FlatMap, known as "bind" in some other languages, is as you said yourself for function composition.

Imagine for a moment that you have some functions like these:

def foo(x: Int): Option[Int] = Some(x + 2)
def bar(x: Int): Option[Int] = Some(x * 3)

The functions work great, calling foo(3) returns Some(5), and calling bar(3) returns Some(9), and we're all happy.

But now you've run into the situation that requires you to do the operation more than once.

foo(3).map(x => foo(x)) // or just foo(3).map(foo) for short

Job done, right?

Except not really. The output of the expression above is Some(Some(7)), not Some(7), and if you now want to chain another map on the end you can't because foo and bar take an Int, and not an Option[Int].

Enter flatMap

foo(3).flatMap(foo)

Will return Some(7), and

foo(3).flatMap(foo).flatMap(bar)

Returns Some(15).

This is great! Using flatMap lets you chain functions of the shape A => M[B] to oblivion (in the previous example A and B are Int, and M is Option).

More technically speaking; flatMap and bind have the signature M[A] => (A => M[B]) => M[B], meaning they take a "wrapped" value, such as Some(3), Right('foo), or List(1,2,3) and shove it through a function that would normally take an unwrapped value, such as the aforementioned foo and bar. It does this by first "unwrapping" the value, and then passing it through the function.

I've seen the box analogy being used for this, so observe my expertly drawn MSPaint illustration: enter image description here

This unwrapping and re-wrapping behavior means that if I were to introduce a third function that doesn't return an Option[Int] and tried to flatMap it to the sequence, it wouldn't work because flatMap expects you to return a monad (in this case an Option)

def baz(x: Int): String = x + " is a number"

foo(3).flatMap(foo).flatMap(bar).flatMap(baz) // <<< ERROR

To get around this, if your function doesn't return a monad, you'd just have to use the regular map function

foo(3).flatMap(foo).flatMap(bar).map(baz)

Which would then return Some("15 is a number")

Electric Coffee
  • 11,733
  • 9
  • 70
  • 131
  • 3
    This answer is clear and concise, answers precisely what the question asks AND has awesome graphics. I wish I could upvote twice – djsecilla Dec 12 '15 at 17:25
  • 2
    This is probably the best description of the concept behind flatMap/bind I have seen. Thanks. – melston Dec 12 '15 at 18:32
  • Nice answer, I think though that the `flatMap:` part of the diagram should show the flatMap mapping then flattering rather than flattering then mapping. – rose specs Feb 16 '22 at 12:00
2

It's the same reason you provide more than one way to do anything: it's a common enough operation that you may want to wrap it.

You could ask the opposite question: why have map and flatten when you already have flatMap and a way to store a single element inside your collection? That is,

x map f
x filter p

can be replaced by

x flatMap ( xi => x.take(0) :+ f(xi) )
x flatMap ( xi => if (p(xi)) x.take(0) :+ xi else x.take(0) )

so why bother with map and filter?

In fact, there are various minimal sets of operations you need to reconstruct many of the others (flatMap is a good choice because of its flexibility).

Pragmatically, it's better to have the tool you need. Same reason why there are non-adjustable wrenches.

Rex Kerr
  • 166,841
  • 26
  • 322
  • 407
1

The simplest reason is to compose an output set where each entry in the input set may produce more than one (or zero!) outputs.

For example, consider a program which outputs addresses for people to generate mailers. Most people have one address. Some have two or more. Some people, unfortunately, have none. Flatmap is a generalized algorithm to take a list of these people and return all of the addresses, regardless of how many come from each person.

The zero output case is particularly useful for monads, which often (always?) return exactly zero or one results (think Maybe- returns zero results if the computation fails, or one if it succeeds). In that case you want to perform an operation on "all of the results", which it just so happens may be one or many.

Chris Shain
  • 50,833
  • 6
  • 93
  • 125
  • This is closer to what I was looking for, I think. I understand the 0/1 element case that `flatMap` seems to cover pretty well. In looking at some of the material I have found online it seems to indicate that it should fulfill kind of the same role as `compose`, though. But that is just an impression I am trying to confirm or dispel. – melston Dec 12 '15 at 03:13
  • Compose is completely different. Compose executes the first (function) argument and passes the results to the second function. What might have you confused is that flatMap is the result of composing the map and flatten functions- first map the results from the input, then flatten those results into a single sequence. – Chris Shain Dec 12 '15 at 04:06
  • However, Brian Beckman explains Haskell's bind operator `>>=` as a monadic composition operator. And, otherwise it seems to be the same as `flatMap`. Was he wrong? – melston Dec 12 '15 at 04:10
  • 1
    No- but flatMap is the superset of functionality in that case. Monadic composition is only for 0 or 1 results, whereas flatMap is for 0 or more. – Chris Shain Dec 12 '15 at 08:29
1

The "flatMap", or "bind", method, provides an invaluable way to chain together methods that provide their output wrapped in a Monadic construct (like List, Option, or Future). For example, suppose you have two methods that produce a Future of a result (eg. they make long-running calls to databases or web service calls or the like, and should be used asynchronously):

def fn1(input1: A): Future[B]  // (for some types A and B)

def fn2(input2: B): Future[C]  // (for some types B and C)

How to combine these? With flatMap, we can do this as simply as:

def fn3(input3: A): Future[C] = fn1(a).flatMap(b => fn2(b))

In this sense, we have "composed" a function fn3 out of fn1 and fn2 using flatMap, which has the same general structure (and so can be composed in turn with further similar functions).

The map method would give us a not-so-convenient - and not readily chainable - Future[Future[C]]. Certainly we can then use flatten to reduce this, but the flatMap method does it in one call, and can be chained as far as we wish.

This is so useful a way of working, in fact, that Scala provides the for-comprehension as essentially a short-cut for this (Haskell, too, provides a short-hand way of writing a chain of bind operations - I'm not a Haskell expert, though, and don't recall the details) - hence the talk you will have come across about for-comprehensions being "de-sugared" into a chain of flatMap calls (along with possible filter calls and a final map call for the yield).

Shadowlands
  • 14,994
  • 4
  • 45
  • 43
  • 1
    Thanks. So `flatMap` really is a way to chain monad-producing functions the same way `map` can be used to chain 'normal' functions. That is, functions that are defined as `A -> B`. – melston Dec 12 '15 at 18:41
  • @melston Yes, that is a reasonable way to think of it. – Shadowlands Dec 12 '15 at 20:40
0

Well, one could argue, you don't need .flatten either. Why not just do something like

@tailrec
def flatten[T](in: Seq[Seq[T], out: Seq[T] = Nil): Seq[T] = in match {
   case Nil => out
   case head ::tail => flatten(tail, out ++ head)
}

Same can be said about map:

@tailrec
def map[A,B](in: Seq[A], out: Seq[B] = Nil)(f: A => B): Seq[B] = in match {
   case Nil => out
   case head :: tail => map(tail, out :+ f(head))(f)
  } 

So, why are .flatten and .map provided by the library? Same reason .flatMap is: convenience.

There is also .collect, which is really just

list.filter(f.isDefinedAt _).map(f)

.reduce is actually nothing more then list.foldLeft(list.head)(f), .headOption is

list match {
    case Nil => None
    case head :: _ => Some(head)
}

Etc ...

Dima
  • 39,570
  • 6
  • 44
  • 70
  • I was hoping for more than TMTOWTDI :) I have seen map defined in terms of `flatMap` and vice versa (for `List`s) so I kind of get that. I was looking for more of a push in some conceptual direction. Though I usually forget about `collect`. Thanks for the reminder. – melston Dec 12 '15 at 03:23