List comprehension vs high-order functions in F#

Question

I come from SML background and feel quite comfortable with high-order functions. But I don't really get the idea of list comprehension. Is there any situation where list comprehension is more suitable than high-order functions on List and vice versa?

I heard somewhere that list comprehension is slower than high-order functions, should I avoid to use it when writing performance-critical functions?

For the example' sake, take a look at Projecting a list of lists efficiently in F# where @cfern's answer contains two versions using list comprehension and high-order functions respectively:

let rec cartesian = function
  | [] -> [[]]
  | L::Ls -> [for C in cartesian Ls do yield! [for x in L do yield x::C]]

and:

let rec cartesian2 = function
  | [] -> [[]]
  | L::Ls -> cartesian2 Ls |> List.collect (fun C -> L |> List.map (fun x->x::C))

score 11 · Accepted Answer · answered Jun 07 '11 at 11:14

Choosing between comprehensions and higher-order functions is mostly a matter of style. I think that comprehensions are sometimes more readable, but that's just a personal preference. Note that the cartesian function could be written more elegantly like this:

let rec cartesian = function  
  | [] -> [[]]  
  | L::Ls -> 
     [ for C in cartesian Ls do for x in L do yield x::C ]

The interesting case is when writing recursive functions. If you use sequences (and sequence comprehensions), they remove some unnecessary allocation of temporary lists and if you use yield! in a tail-call position, you can also avoid stack overflow exceptions:

let rec nums n = 
  if n = 100000 then []
  else n::(nums (n+1))
// throws StackOverflowException
nums 0 

let rec nums n = seq {
  if n < 100000 then
    yield n
    yield! nums (n+1) }
// works just fine
nums 0 |> List.ofSeq

This is quite an interesting pattern, because it cannot be written in the same way using lists. When using lists, you cannot return some element and then make a recursive call, because it corresponds to n::(nums ...), which is not tail-recursive.

Daniel · Answer 2 · 2011-06-07T17:40:54.480

Looking at the generated code in ILSpy, you can see that list comprehensions are compiled to state machines (like methods using yield return in C#), then passed to something like List.ofSeq. Higher-order functions, on the other hand, are hand-coded, and frequently use mutable state or other imperative constructs to be as efficient as possible. As is often the case, the general-purpose mechanism is more expensive.

So, to answer your question, if performance is critical there is usually a higher-order function specific to your problem that should be preferred.

score 0 · Answer 3 · answered Feb 10 '17 at 08:57

Adding to Tomas Petricek's answer. You can make the list version tail recursive.

let nums3 n =
    let rec nums3internal acc n = 
        if n = 100000 then
            acc
        else
            nums3internal (n::acc) (n+1) //Tail Call Optimization possible

    nums3internal [] n |> List.rev

nums3 0

With the added benefit of a considerable speedup. At least when I measured with the stopwatch tool I get. (nums2 being the algorithm using Seq).

Nums2 takes 81.225500ms
Nums3 takes 4.948700ms

For higher numbers this advantage shrinks, because List.rev is inefficient. E.g. for 10000000 I get:

Nums2 takes 11054.023900ms
Nums3 takes 8256.693100ms

List comprehension vs high-order functions in F#

3 Answers3

Linked