Access pixel values without loading image in memory for large images

Question

I need to compute the mean value of an image using CImg library like this:

int i = 0;
float mean = 0;
CImg<float> img("image.cimg");  
float *ptr = img.data(); //retrieves pointer to the first value
while(i<img.width()*img.height()*img.spectrum()){
    mean += *(ptr+i);
    ++i;
}
std::cout << "mean: " << mean/i << std::endl;

I know that img.mean() would do the trick, but here I want to do it in a low-level way.

When the size of the image increases too much, the 3rd line in my code consumes too much resources of my computer because according to the documentation it is storing all the image pixels in a memory buffer at the same time.

I thought about an even lower level solution, using the system calls open() and read() as follows:

int i = 0;
int k = WIDTH*HEIGHT*SPECTRUM; //assuming this values are known
float mean = 0, aux;
int fd = open("image.cimg", O_RDONLY);
while(i<k){ 
    read(fd, &aux, sizeof(float));
    mean += aux; 
    ++i;
}
close(fd);
std::cout << "mean: " << mean/i << std::endl;

But the results obtained now don't make any sense. I wonder if this solution makes any sense at all, if the image is stored at the disk at the same way it is when loaded at the memory, and if at the end this solution would save time and memory or not.

Compound data usually contains a header, describing the information. The `CImg`-c'tor knows how to read that, and provides the respective numbers (`img.width()`, etc.). Your naive solution reads the header like it were image data, and makes other assumptions about layout that may not be true. — IInspectable, Nov 26 '13 at 17:51
In addition to @IInspectable: define "stored at the disk at the same way it is when loaded at the memory". You can't answer that without knowing how CImg stores its data internally, but a good bet is the answer is "no". The reading routine interprets stuff like width, height, bits per pixel, and so on and hides these details from the programmer through its class. If you can get hold of a description of your "cimg" files (a type I am not familiar with), you can write a class yourself and add, for example, a one-line only read function. That takes time but you get to use far less memory. — Jongware, Nov 26 '13 at 18:47

score 1 · Answer 1 · answered Sep 30 '16 at 10:18

The problem is the second line of your code because you have made mean (although it would be better named sum) a simple float. As each pixel in your image is also a float, you will run into problems if your image is, say 10,000x10,000 because you would be trying to store the sum of 100M floats in a float.

The easiest solution is to change line 2 to:

double mean=0;

As an alternative, you can calculate the mean incrementally as you go along without it overflowing like this:

float mean = 0;
int i = 1;
while(...){
  mean+= (x - mean)/i;
  ++i;
}

By the way, if you have really large images, may I recommend vips, it is very fast and very efficient, for example, if I create a 10,000x10,000 pixel TIF and ask vips to average it from the command line:

time vips avg image.tif --vips-leak
0.499994

memory: high-water mark 7.33 MB

real    0m0.384s
user    0m0.492s
sys     0m0.233s

You can see it take 0.4 seconds and peaks out at 7MB memory usage.

Access pixel values without loading image in memory for large images

1 Answers1