0

I'm working with an ENVI image file (.hdr labbeled) with a interleave of the type bsq (band sequential). I'm trying to do a principal component transformation of an image about 350MB large. In order to do that I had to proccess it separately in smaller memory blocks. The code below works as expected if the file can be worked with in just one block of memory. However, if it takes more than one iterarion, only the last block comes out as expected, i.e, all the previous blocks are written as if they were a repetition of the same pixel over and over again.

extern void pca_xform(envi_header file){

    #define BLOCK_SIZE 3000000

          /*Calculates covariance_matrix, eigen_matrix, mean_vector 
    and allocates pixel_vector and xform_pixel_vector*/

      int size = (file.lines)*(file.samples), i;

      printf("Transforming\n");

      int n = size/BLOCK_SIZE;
      int r = size - ((BLOCK_SIZE)*n);
      int block;

      for(i=0,block=0;i<n+1;++i,++block){
        int actual_size = BLOCK_SIZE;
        if(i==n){
          actual_size = r;
        }
        int k;
        io_bsq_read_pixel(file, i*BLOCK_SIZE, actual_size, pixel_vector);
        for(k=0;k<actual_size;++k){
          pca_normalize_pixel_vector(mean_vector, pixel_vector[k]);
        }
        for(k=0;k<actual_size;++k){
          pca_xform_pixel_vector(file, eigen_matrix, pixel_vector[k], xform_pixel_vector[k]);
        }
        io_bsq_write_pixel(file, i*BLOCK_SIZE, actual_size, xform_pixel_vector);
      }

      return;
    }

Here is the writing function. 

extern void io_bsq_write_pixel(envi_header file, int start, int number, gsl_vector** pixel_vector){
  FILE* fp = fopen(file.new_filename, "wb");
  if(!fp){
    fprintf(stderr, "Failed to open target: %s\n", file.new_filename);
    exit(EXIT_FAILURE);
  }
  int size = (file.samples) * (file.lines);
  int i, j;
  double d;
  for(i=0;i<file.bands;++i){
    fseek(fp, sizeof(double)*((size*i)+start), SEEK_SET);
    for(j=0;j<number;++j){
      d = gsl_vector_get(pixel_vector[j], i);
      fwrite(&d, sizeof(double), 1, fp);
    }
  }
  fclose(fp);
  return;
}

I've come to the conclusion that the unexpected behaviour is due to this function itself or for some inappropriate call to it in the pca_xform function. To do this, I've simply used the following code insted, which writes the pixels sequentially (bip interleave).

for(i=0;i<size:++i){
   gsl_vector_fwrite(fp, xform_pixel_vector[i]);
 }

I would like, however, to keep my output file as a band sequential one. I've used a lot of time trying to find a solution, tweaking the code here and there, but the problem's solution still eludes me.

lfmc
  • 31
  • 1
  • 7
  • I find the `fseek()` suspicious. If you're writing sequentially, why are you seeking at all? – EOF Apr 04 '16 at 20:35
  • @EOF I'm using the `fseek()` so that I can find the correct position to start writing that band. Say I have a 2-banded 10MB image as output. This allows me to write the first 2,5MB (block 1, for instance) of band 1, then go to band to 2 and write the other 2,5MB. So I left blank spaces in the correct position to write the second block of 5MB. – lfmc Apr 04 '16 at 20:40
  • I know this would prevent compile, and is not the source of your problem.... On the final code snippet you need a closing `)`. – ryyker Apr 04 '16 at 20:54
  • @ryyker Thanks for pointing it out. I've corrected it. – lfmc Apr 04 '16 at 21:01
  • @lfmc - Did you ever solve your problem? – ryyker Apr 05 '16 at 20:54
  • @ryyker Yes, I did. I was careless in the way I was opening the file. – lfmc Apr 06 '16 at 22:24

1 Answers1

0

I finally managed to find the source of the problem. The problem was now as I was so inclined to believe in the position of the writing procedures in the file. It was still, however in the io_bsq_write_pixel. Each time I called the function it would subsequentally call fwrite(file.new_filename, "w"). Opening the file in "w" mode erased all my previous work. In order to fix this, I initialized the file before calling pca_xform and then changed the io_bsq_write_pixel

    extern void io_bsq_write_pixel(envi_header file, int start, int number, gsl_vector** pixel_vector){
  FILE* fp = fopen(file.new_filename, "r+b");
  if(!fp){
    fprintf(stderr, "Failed to open target: %s\n", file.new_filename);
    exit(EXIT_FAILURE);
  }
  int size = (file.samples) * (file.lines);
  int i, j;
  double d;
  for(i=0;i<file.bands;++i){
    fseek(fp, sizeof(double)*((size*i)+start), SEEK_SET);
    for(j=0;j<number;++j){
      d = gsl_vector_get(pixel_vector[j], i);
      fwrite(&d, sizeof(double), 1, fp);
    }
  }
  fclose(fp);
  return;
}

This way, using "r+b" and having initialized the file first, I guarantee that it will open the file for a simple update, without erasing my previous work.

lfmc
  • 31
  • 1
  • 7