From what I understood in Wikipedia's explanation of quicksort's space complexity, quicksort's space complexity comes from its recursive nature. I'm curious as to whether it's possible to implement quicksort non-recursively and, in doing so, implement it with constant space complexity.
Asked
Active
Viewed 5,552 times
11
-
@trumpetlicks: O(1) **space complexity**. – j_random_hacker Jul 12 '12 at 15:33
-
@j_random_hacker - good point, Im thinking algorithmic complexity :-) – trumpetlicks Jul 12 '12 at 15:34
-
yes, you just have to pick the pivot elements in such a way that you are able to predict the partition sizes. – salva Jul 12 '12 at 15:42
-
3@Daniel: It's been proven that you can't do a sort based on comparisons with less than O(N log N) time complexity. – Jerry Coffin Jul 12 '12 at 15:46
-
@salva: Median of medians will do that -- but it requires O(log N) space to pick those pivots... – Jerry Coffin Jul 13 '12 at 21:26
-
@JerryCoffin: you can use the [quickselect](http://en.wikipedia.org/wiki/Selection_algorithm) that when implemented iteratively requires O(1) space. Worst case complexity is O(N**2), but you already had that with the naive quicksort. – salva Jul 14 '12 at 08:09
3 Answers
19
Wikipedia is not always wrong. And, as the section suggests, there is a way to do the quicksort, or something similar, using constant space. One important point. Quicksort itself could be defined as a recursive partitioning algorithm. If so, then by definition it will require O(n) stack space. However, I'm assuming that you are not using such a pedantic definition.
Just a quick review of how the partitioning works. Given an array, a starting point and an ending point, a partition value is chosen. The data elements in the array are then split so everything less than the partition value is on the left and everything greater is on the right. A good way of doing this is by starting at each end, finding the first value that doesn't belong, and swapping them. This, by the way, uses constant space.
So, each step of the algorithm is going through the array. Let's remember this fact.
Now, we can make an interesting observation. If we do the recursive partitioning in a depth-first fashion, then we only have to store the end points of each range. On the way down, the left edge of the array is always the beginning. The end point gets successively close to the beginning, until there are just two elements that can be swapped, or not. At this point, the beginning moves over two slots, but we don't know the end. So, look up the end and continue the process. Then at the next step "up", we need the next end point, and so on.
The question is: Can we find the end by some means other than storing the actual value in a stack?
Well, the answer is "yes".
Each step in the recursive partitioning algorithm reads through all the data. We can do some additional calculations on the data. In particular, we can calculate the largest value and the second largest value. (I would also calculate the smallest value as well, but that is an optimization.).
What we do with the values is mark the ranges. On the first split, this means putting the second largest value at the split point and the largest value at the end of the range. On the way back up the tree, you know where the range starts. The end of the range is the first value larger than that value.
Voila! You can move up the "recursion" tree without storing any data. You are just using the data as presented.
Once you have accomplished this, you simply need to change the algorithm from a recursive algorithm to a while loop. The while loop rearranges the data, setting a starting point and stopping point at each step. It chooses a splitter, splits the data, marks the starting and ending point, and then repeats on the left side of the data.
When it has gotten down to the smallest unit, it then check whether it is done (has it reached the end of the data). If not, it looks at the data point one unit over to find the first marker. It then goes through the data to look for the end point. This search, by the way, is equivalent in complexity to the partitioning of the data, so it does not add to the order of complexity. It then iterates through this array, continuing the process until it is done.
If you have duplicates in the data, the process is slightly more complex. However, if there are log(N) duplicates, I would almost argue for removing the duplicates, sorting the data using the remaining slots as a stack, and then incorporating them back in.
Why this is quicksort. The quicksort algorithm is a partition exchange algorithm. The algorithm proceeds by choosing a splitter value, partitioning the data on the two sides, and repeating this process. Recursion is not necessary, as Jeffrey points out in his answer. It is a great convenience.
This algorithm proceeds in exactly the same way. The partitioning follows the same underlying rule, with smaller records on the left and larger records on the right. The only difference is that within each partition, particular values are chosen to be on the edges of the partition. By careful placement of these values, no additional "per-step" storage is needed. Since these values belong in the partition, this is a valid partition according to the quicksort principal of partition-and-repeat.
If one argues that a quicksort must use recursion, then this would fail that strict test (and the answer to the original question is trivial).
Gordon Linoff
- 1,242,037
- 58
- 646
- 786
4
It's entirely possible to implement it non-recursively, but you do that by implementing a stack separate from the normal function call/return stack. It may save some space by only storing the essential information instead of a lot of (mostly identical) function return addresses, but its size is still going to be logarithmic, not constant.
Quicksort Definition
Since there's been discussion about whether (for example) the algorithm cited by @Gordon Linoff in his answer is really a Quicksort, I'll refer to C.A.R. Hoare's paper describing Quicksort, which seems to me the most authoritative source available about what does or does not constitute a Quicksort. According to his paper:
Meanwhile, the addresses of the first and last items of the postponed segment must be stored.
While it's not too much of a stretch to store something that's (more or less) equivalent to an address (e.g., an index) rather than an actual index, it seems to me that when the description of the algorithm directly states that you must store an address, an algorithm that does not store an address or anything even roughly equivalent to it, is no longer an implementation of that same algorithm.
Reference
Jerry Coffin
- 476,176
- 80
- 629
- 1,111
-
fwiw, there is a way to do quicksort in-place. I would imagine that you could efficiently save pivots and the like. – Dennis Meng Jul 12 '12 at 15:44
-
@DennisMeng: Yes, you can efficiently save pivots in a stack. Assuming you always sort the smaller partition first, the number of pivots you have to save is limited to O(log N). IOW, the amount of extra space you need is O(log N). – Jerry Coffin Jul 13 '12 at 03:36
-
This answer is incorrect. See the version below. You can store the ranges in the data, without affecting the complexity of the algorithm. – Gordon Linoff Jul 13 '12 at 13:19
-
@GordonLinoff: While your answer outlines an *interesting* algorithm, calling it a Quicksort is *quite* a stretch. It would be just about as accurate for me (or whomever) to call a heapsort a "quicksort that doesn't require extra storage" (which is to say that your algorithm is about as much (if not more) like a heapsort as a real quicksort). – Jerry Coffin Jul 13 '12 at 14:01
-
1@JerryCoffin. The algorithm that I describe partitions the data, then sorts each partition. The difference with the classical quicksort is that it does this via a loop rather than recursion (which is necessary as per the question). The other difference is that it tweaks the placement of two values per partition, instead of storing information in a stack. I don't see the relationship to heap sort at all. This algorithm has exactly the same issues choosing splitters and the same performance characteristics of quicksort. – Gordon Linoff Jul 13 '12 at 14:16
-
1@GordonLinoff: Well, I suppose you can hold whatever opinion you want to. It's probably true that if you asked 10 experienced programmers: "If I wanted to, could I reasonably call this a quicksort?", at least half would probably at least sort of agree that you could. On the other hand, if you said "what's the name of this sort", I'd be surprised if more than one said it was a quicksort (and if even one did, it would probably be because they didn't read it very carefully). It's not a reasonable basis for claiming my answer is wrong (or the apparent down-vote). – Jerry Coffin Jul 13 '12 at 21:23
3
Branislav Ďurian presented a constant-space version of Quicksort in 1986. See his paper "Quicksort without a stack". In J. Gruska, B. Rovan, and J. Wiedermann, editors, Proceedings of the Mathematical Foundations in Computer Science, volume 233 of Lecture Notes in Computer Science, pages 283–289. Springer-Verlag, 1986.
Several other authors followed up on this. You can look for Bing-Chao and Knuth (1986); Wegner (1987); Kaldewaij and Udding (1991); Gries (1994).
wstomv
- 761
- 1
- 6
- 13