7

Is it possible to implement a tail recursive version of the quick sort algorithm (via the continuation pattern)? And if it is, how would one implement it?

Normal (not optimized) version:

let rec quicksort list =
 match list with
 | [] -> []
 | element::[] -> [element]
 | pivot::rest -> let ``elements smaller than pivot``, ``elements larger or equal to pivot``= 
                    rest |> List.partition(fun element -> element < pivot)
                  quicksort ``elements smaller than pivot`` @ [pivot] @ quicksort ``elements larger or equal to pivot``
ghilesZ
  • 1,502
  • 1
  • 18
  • 30
Yet Another Geek
  • 4,251
  • 1
  • 28
  • 40
  • 4
    The second case is not strictly necessary, as the third case will do the right thing for it. I wouldn't call a quicksort made tail-recursive against its nature with continuations "optimized". Everything you don't allocate in the stack will just go in the heap, you have to realize. – Pascal Cuoq Apr 12 '11 at 11:00
  • It is not necessary, but as I do not know how it would be compiled I thought it would save a few function calls. (even though it is not that important because of O(1) run time) – Yet Another Geek Apr 12 '11 at 11:03
  • I was just referring to the normal version as not optimized. I do realize there might be more memory usage anyway. There are also some limitation with the stack which does not exist in the heap, and thus it can help reducing stackoverflows. – Yet Another Geek Apr 12 '11 at 11:05
  • Do you have a real-world application for this? List.sort is not supposed to stack overflow in F#. In OCaml, I think it is faster to convert to array and sort the array (when you have a huge list). – Laurent Apr 12 '11 at 11:22
  • I am just curious if it is possible to do so. – Yet Another Geek Apr 12 '11 at 11:28
  • 2
    What you haven't isn't exactly a quicksort -- the pivot should be chosen randomly. What you have will exhibit worst case O(n^2) behavior on sorted or nearly-sorted input. You might get a faster sort using one of these strategies: 1) use a merge sort instead, or 2) dump the list in an array, sort said array, convert back to list. One last thing: `list1 @ [x] @ list2` is slow, use `list1 @ x::list2` instead. – Juliet Apr 12 '11 at 21:19
  • If one assume that the list is shuffled, and therefore should be sorted, choosing the first element in the list is as good as choosing an element inside the pivot. A common approach is to pick three numbers(e.g. first, middle and last) and choosing the number which is between the greatest and smallest of these three numbers. It will only run badly if the list is already sorted, but in a linked list like F# another approach would require much more work. As for the last thing it was only for demonstration propose as one should never use this version of quicksort. – Yet Another Geek Apr 13 '11 at 07:56
  • 2
    @Juliet: This algorithm is not the real quicksort because it is not using in-place partitioning to reduce IO and, consequently, it is extremely *slow*. – J D Jun 25 '11 at 15:46
  • @Jon I assume that List.partition was optimized as it is part of the standard library. Of course the standard way of doing it would be to convert the list to an array and do an in-place edit, but that doesn't seem like a very _functional_ way of doing it. On an other note I was not using this for speed but to learn more about CPS. – Yet Another Geek Jun 25 '11 at 17:15
  • 2
    @Geek: Quicksort is an inherently impure algorithm because it is based upon in-place partitioning in order to minimize IO so it doesn't make sense to create a pure algorithm that looks similar but sacrifices the efficiency and still call it "quicksort". By all means use it to learn about CPS. I'm just warning that it is misleading to call this "quicksort". – J D Jul 03 '11 at 21:45

3 Answers3

14

Direct style:

let rec quicksort list =
    match list with
    | [] -> []
    | [element] -> [element]
    | pivot::rest ->
        let left, right = List.partition (fun element -> element < pivot) rest in
        let sorted_left = quicksort left in
        let sorted_right = quicksort right in
        sorted_left @ [pivot] @ sorted_right

My first, naive translation is very similar to Laurent's version, except indented a bit weirdly to make apparent that calls with continuations are really a kind of binding:

let rec quicksort list cont =
    match list with
    | [] -> cont []
    | element::[] -> cont [element]
    | pivot::rest ->
        let left, right = List.partition (fun element -> element < pivot) rest in
        quicksort left (fun sorted_left ->
        quicksort right (fun sorted_right ->
        cont (sorted_left @ [pivot] @ sorted_right)))
let qsort li = quicksort li (fun x -> x)

Contrarily to Laurent, I find it easy to check that cont is not forgotten: CPS functions translated from direct style have the property that the continuation is used linearily, once and only once in each branch, in tail position. It is easy to check that no such call was forgotten.

But in fact, for most runs of quicksort (supposing you get a roughly logarithmic behavior because you're not unlucky or you shuffled the input first), the call stack is not an issue, as it only grows logarithmically. Much more worrying are the frequent calls to @ wich is linear in its left parameter. A common optimization technique is to define functions not as returning a list but as "adding input to an accumulator list":

let rec quicksort list accu =
    match list with
    | [] -> accu
    | element::[] -> element::accu
    | pivot::rest ->
        let left, right = List.partition (fun element -> element < pivot) rest in
        let sorted_right = quicksort right accu in
        quicksort left (pivot :: sorted_right)
let qsort li = quicksort li []

Of course this can be turned into CPS again:

let rec quicksort list accu cont =
    match list with
    | [] -> cont accu
    | element::[] -> cont (element::accu)
    | pivot::rest ->
        let left, right = List.partition (fun element -> element < pivot) rest in
        quicksort right accu (fun sorted_right ->
        quicksort left (pivot :: sorted_right) cont)
let qsort li = quicksort li [] (fun x -> x)

Now a last trick is to "defunctionalize" the continuations by turning them into data structure (supposing the allocation of data structures is slightly more efficient than the allocation of a closure):

type 'a cont =
  | Left of 'a list * 'a * 'a cont
  | Return
let rec quicksort list accu cont =
    match list with
    | [] -> eval_cont cont accu
    | element::[] -> eval_cont cont (element::accu)
    | pivot::rest ->
        let left, right = List.partition (fun element -> element < pivot) rest in
        quicksort right accu (Left (left, pivot, cont))
and eval_cont = function
  | Left (left, pivot, cont) ->
    (fun sorted_right -> quicksort left (pivot :: sorted_right) cont)
  | Return -> (fun x -> x)
let qsort li = quicksort li [] Return

Finally, I chose the function .. fun style for eval_cont to make it apparent that those were just pieces of code from the CPS version, but the following version is probably better optimized by arity-raising:

and eval_cont cont accu = match cont with
  | Left (left, pivot, cont) ->
    quicksort left (pivot :: accu) cont
  | Return -> accu
gasche
  • 31,259
  • 3
  • 78
  • 100
  • Why would the allocation of data structures be more efficient the allocation of a closure? It should be almost the same, no? – Laurent Apr 12 '11 at 15:53
  • I have tried your last function, it's slightly faster than the "naive translation", but there's not a big difference. Both versions are still extremely slow sorting functions (as was the original one). Anyway, it's still a fun exercise. – Laurent Apr 12 '11 at 16:32
  • 1
    @Laurent I would be more interested in a comparison of the direct-style accumulator-using function wrt. the simplest one. – gasche Apr 12 '11 at 16:35
  • 1
    @Laurent, [Reynolds defunctionalization](http://en.wikipedia.org/wiki/Defunctionalization) is the last step in compiling a higher-order functional language to code that doesn't require the machinery necessary to implement closures. This in fact allows to compile easily such languages as Scheme that have first-class continuations down to C or assembler. –  Apr 13 '11 at 12:45
3

Quick attempt, seeems to work:

let rec quicksort list cont =
    match list with
    | [] -> cont []
    | element::[] -> cont [element]
    | pivot::rest ->
        let ``elements smaller than pivot``, ``elements larger or equal to pivot`` =
            rest |> List.partition (fun element -> element < pivot)
        quicksort ``elements smaller than pivot``
            (fun x -> quicksort ``elements larger or equal to pivot`` (fun y -> cont (x @ [pivot] @ y)))

> quicksort [2; 6; 3; 8; 5; 1; 9; 4] id;;
val it : int list = [1; 2; 3; 4; 5; 6; 8; 9]

Edit:

Of course, this code is highly inefficient. I hope nobody will use it in real code. The code was not difficult to write, but continuations might be difficult to read and can be error-prone (it's easy to forget a call to cont). If you want to play more, you can write a continuation monad (Brian wrote a blog post about it).

Laurent
  • 2,951
  • 16
  • 19
3

Continuation monad (stolen from here) can also be used (usually makes code more readable):

type ContinuationMonad() =
    // ma -> (a -> mb) -> mb
    member this.Bind (m, f) = fun c -> m (fun a -> f a c)
    // a -> ma
    member this.Return x = fun k -> k x
    // ma -> ma
    member this.ReturnFrom m = m
let cont = ContinuationMonad()

// Monadic definition of QuickSort
// it's shame F# doesn't allow us to use generic monad code
// (we need to use 'cont' monad here)
// otherwise we could run the same code as Identity monad, for instance
// producing direct (non-cont) behavior
let rec qsm = function
     |[]    -> cont.Return []
     |x::xs -> cont {
        let l,r = List.partition ((>=)x) xs
        let! ls = qsm l 
        let! rs = qsm r
        return (ls @ x :: rs) }

// Here we run our cont with id
let qs xs = qsm xs id     

printf "%A" (qs [2;6;3;8;5;1;9;4])
Ed'ka
  • 6,595
  • 29
  • 30