Memory-efficient way of computing the median of a large data set?

Question

If one computer can only hold 1 million numbers, how to find out the median number from 100 million numbers?

http://stackoverflow.com/questions/1311764/is-it-possible-to-calculate-median-of-a-list-of-numbers-better-than-on-log-n — Ron, Sep 25 '09 at 02:29
This is a valid programming related question, how to compute the median in a memory efficient way. It just comes along as a puzzle. — starblue, Sep 25 '09 at 07:04

DigitalRoss · Answer 1 · 2009-09-25T02:13:02.363

3

Hopefully, the real problem was "how do I do an external sort"? (If this is homework...I want to help in the right way. :-)

edited Sep 25 '09 at 02:13

answered Sep 25 '09 at 02:05

DigitalRoss

This is what I thought. :) But I'm not sure it's the correct answer, so I asked here. – Stephen Hsu Sep 25 '09 at 02:11
1

There has got to be a way to do this with the literal constraint that the device can only store 1 million numbers. Using the external sort seems like cheating. Now I'm gonna be up all night thinking about this. – JohnFx Sep 25 '09 at 02:28
Heh, I wondered that myself. It's a really good question. – DigitalRoss Sep 25 '09 at 17:47

score 3 · Answer 2 · answered Sep 25 '09 at 02:06

3

Reduce the problem to a more difficult one: sort the 100 million numbers using merge sort Then, take the 50 millionth element.

answered Sep 25 '09 at 02:06

Pascal Cuoq

But the computer can hold 1 million numbers only, how can I find the 50 millionth one? – Stephen Hsu Sep 25 '09 at 02:12
1

On the tape (oh, right, this is no longer the 80s. I meant "on the disk"), at the 50 millionth position. You have storage for your 100M elements, right? Because if you don't (elements read from a stream) the exercice can be proved impossible by a counting argument. – Pascal Cuoq Sep 25 '09 at 02:15
1

It is not correct for 100 million numbers to take 50 millionth element, because 100 million is even number, so one has to take mean of 50 millionth and 50 millionth + 1 elements – Timofey Feb 06 '12 at 11:03

score 1 · Answer 3 · answered Sep 25 '09 at 02:07

1

Using 101 computers and a sort-merge just like a database.

answered Sep 25 '09 at 02:07

tster

Lol. This answer should feature as the best programmer joke! – Ashwin Sep 25 '09 at 02:13
I'll take it as one of my answers.:) – Stephen Hsu Sep 25 '09 at 02:14

score 0 · Answer 4 · answered Sep 25 '09 at 02:19

0

Find the middle million numbers and then report the median of them. (Hmmm, now how to find those middle million numbers...)

answered Sep 25 '09 at 02:19

PaulMcG

4 Answers4