0

This is one of the requirement i came across my work. We have a (2 power 32) contiguous 4294967296 integers allocated as an array in memory whose function is to provide mapping in another table. Some of the entries gets written more often than the other. We want to track the hot spots and provide an approximate histogram. The catch is that, this is going to be implemented in firmware and not much memory can be used.

information: The mapping is for scsi lba from host to lbas on the target probably drives or flash memory.

Lets say we have 1 MB space to handle the meta data required to track hot-cold information. How can we use this efficiently other than just bit mapping which shows whether it is written or not. WE can extent and have a mathematical extension on how accurate the data we collect is based on how larget the memory is used for tracking.

bicepjai
  • 1,615
  • 3
  • 17
  • 35
  • well according to your requirements, you need to have each entry separate in the histogram or your information isn't really useful. As such you need to store information about 4G elements. This right there is probably too much memory for your firmware application. I'd suggest picking N most-used elements and storing the counts for those N. It will provide you with enough information about hotspots without using much memory. However, this does make the histogram not possible. – im so confused Sep 26 '12 at 22:08
  • OR you widen your histogram bins. Which will show good hotspots for a range, but will defeat the purpose as i'm sure entry X and X+1 have no relation in terms of number of writes – im so confused Sep 26 '12 at 22:09
  • You could maintain write counts for the two halves of the entire set, four quarters, eight eighths, etc, and dynamically zoom deeper into the fractions with the highest counts. – Alexey Frunze Sep 26 '12 at 23:12
  • The most simple and efficient way would be to make an array of required block range size and track writes that happend in block range. But i wanted to know if there are efficient ways to accomplish the task. i updated the question for little more details Alex, could you elaborate your answer ? – bicepjai Sep 26 '12 at 23:16

0 Answers0