0

I have a dataset comprised of n unsorted tuples representing numbers (let's say specific color codes) and their frequency (number of times of appearance).

I want to find an exact median of the numbers with worst case complexity of O(n).

For example:

dataset: {(5000, 8000), (25000, 4000), (9, 9000)} median: 5000

dataset: {(7000, 4), (23000, 400), (3000, 9000), (2500, 12000), (19000, 350), (500, 9000)....} median: ?

Failed attempts so far:

  1. "Decompress" the list (so that it looks like this: {7000, 7000, 7000, 7000, 23000, 23000...}) and then sort it. Problem is - it takes Ω(nlogn), and probably more since the frequencies can be very large and doesn't have any upper bound.
  2. Try to use QuickSelect over the data. To ensure O(n) time complexity we must guaranty good pivot selection. To do so I thought about Median of medians (supposedly O(n)) with the data - but I couldn't figure out how to do that without decompressing, thus making it potentially more than O(n).

Is there a way to manipulate the tuple list so that it wouldn't be decompressed and still use median of medians or another way to find the median?

End note: I don't want to assume anything about the dataset - amount of tuples, confined range of numbers/frequencies, etc).

bashstash
  • 23
  • 6
  • If you can settle for an approximation: [median of medians](https://en.wikipedia.org/wiki/Median_of_medians) algorithm runs in O(n) time – Dillon Davis May 13 '22 at 17:45

1 Answers1

0

Use quickselect on the values, and only pay attention to the frequencies in determining which half to keep.

Your ideal pivot is one that splits the list values in half. Because that will cut the work of the next pass in half. Where that split happens to in the whole dataset doesn't particularly. Because your goal is to get it down to the one value you want, and then you're done.

This means that for median of medians you can ignore frequencies entirely in selecting a pivot. Then pay attention to frequencies when deciding which side of the pivot to keep. And ignore frequencies again while selecting the next pivot.

btilly
  • 43,296
  • 3
  • 59
  • 88
  • But what if my data is somewhat close to this: {(7000, 4000), (23000, 10), (24000, 10), (25000, 10), (21000, 10), (22000, 10)} Wouldn't that result in bad choice of pivot? – bashstash May 13 '22 at 19:15
  • @bashstash As long as the pivot keeps splitting the values (not data, just values) fairly evenly, you'll soon get down to one value. And then you have your answer. – btilly May 13 '22 at 21:24
  • @bashstash In other words the frequency of the values doesn't matter. Getting rid of many values each time is good, how much data that represented doesn't matter. – btilly May 13 '22 at 22:26
  • 1
    The only purpose of "median of medians" is to satisfy proofs. In practice, a randomly-selected pivot works just fine, for very little effort. – rici May 13 '22 at 23:04