5

I'm a student in Electrical Engineering.

As an assigment I need to implement the back projection algorithm used in medical imaging to form in image. To calculate the final image a lot information is calculated and kept in a vector. At a certain desired resolution (256*256 pixels) of the final image, the program crashes as I run out of RAM, so I decided to write this information to 90 text files.

I use ofstream to write these file.

The time needed to calculate this information and then storing it in the vector is:

  • output resolution 64x64 : 13,4s
  • output resolution 128x128 : 140s

Writing this information in .txt files:

  • 64x64 156s (1,25 MB/file)
  • 128x128 1400s (5MB/file)

Code writing to files:

ofstream file;
    for(k = 0; k < 90; k++)
        {    
        oss.str(""); //string stream
        oss << "rec\\reconstruction_matrix_step"<< k << ".txt" ; // per step other file
        filename = path;
        filename.append(oss.str());
        file.open(filename.c_str());
        double weight;
          for( l = 0; l < resolution; l ++)
          {

           bestand << "Begin " << l << endl;
           l_border = - WIDTH*(resolution*1.0/2.0 - l);
           r_border = - WIDTH*(resolution*1.0/2.0 - l) + WIDTH;

           for(i = 0; i < resolution; i++)
           {
              for(j = 0; j < resolution; j++)
              {  
                     file << getSurface(pixels[i][j], l_border, r_border) << "\t";
              }
                file << "\n";
           }
           file << "End" << l << "\n\n\n";
          }
            file.close();
        }

When I use a vector, getSurface(pixels[i][j], l_border, r_border) is put in a vector instead of being written to a file.

Is there any way I can speed up this proces?

Joe
  • 41,484
  • 20
  • 104
  • 125
pivu0
  • 138
  • 1
  • 10
  • I don't see anything wrong with your approach, but there may be some inefficencies in your operator<< for the surface that we can't see. – Vaughn Cato Nov 11 '12 at 21:46
  • 1
    The nature of the differential in the data set your writing should not introduce the variance in timing you describe. 4x-data should not take 10x-time to write *unless the data generation routines don't scale*. Suggest you look at what getSurface() is doing. – WhozCraig Nov 11 '12 at 21:49
  • 2
    I would honestly eliminate the file-writes entirely except at the very end. a 5MB stringstream (which you're using anyway at the head of this code to format your header) is not unreasonable. Dump *everything* to that, then dump that to your output file in one shot. That will all-but-ensure your generator to be the only culprit left. – WhozCraig Nov 11 '12 at 21:51
  • Getsurface is a very long method that really only checks a lot of conditions. The problem is that when writing getSurface to files, it takes 10x as long as just putting getSurface in a vector As I'm not a computer scientist, the only method I've learned to write files is using ofstream. And maybe I'm not even using it in a efficient way. I really don't know. – pivu0 Nov 11 '12 at 21:53
  • 2
    Based on what you are saying the problem is unlikely to be the file writes (although formatting floating point numbers isn't particular fast): If you see a non-linear increase with the size of the outputs, something else is causing this! My suspicion is that inside `getSurface()` some data structure (e.g. a `std::vector`) is copied where it should really just be referenced. – Dietmar Kühl Nov 11 '12 at 21:57
  • This is the header of getSurface: double getSurface(const pixel &pxl,double border_laser_l,double border_laser_r) So besides the 2 doubles nothing is copied. Maybe it could be the formating of doubles. If that is the problem, I can't really make the program any faster I assume? – pivu0 Nov 11 '12 at 22:04
  • Writing all the text to a stringstream and then writing the entire file at once makes it go +- 7% faster. It's not much but its something :) – pivu0 Nov 11 '12 at 22:06
  • WhozCraig, actually going from 64x64 to 128x128 is eight times as much information. So the scaling is right. – pivu0 Nov 11 '12 at 22:09
  • 1
    Run profiler. This is the tool to resolve issues like this one. – SChepurin Nov 12 '12 at 09:10

2 Answers2

2

If bestand is a copy leftover and the same as file, I would replace endl with '\n'. std::endl flushes your output stream and ruins any throughput gained by iostream buffering.

If this is not your problem, I recommend profiling your program. With profiling, you replace guessing with numbers, on which you can base further action.

Olaf Dietsche
  • 72,253
  • 8
  • 102
  • 198
  • Actually the code is in dutch, so I changed the names of the variables in English, but I forgot to change this ;). I will try profiling and will tell you the result this evening! Thanks for the tip – pivu0 Nov 13 '12 at 11:50
  • Try the `endl` change first. This should already boost your throughput. – Olaf Dietsche Nov 13 '12 at 11:55
  • This makes it about 1% faster. Thanks for the tip! – pivu0 Nov 13 '12 at 16:52
  • @pivu0 That's disappointing, I hoped for a lot more. In my tests `\n` vs `std::endl`, `\n` was a factor of 10 faster, resulting in about 65 MB/sec. – Olaf Dietsche Nov 13 '12 at 16:59
  • Writing to binary files made it 10 times as fast. – pivu0 Nov 13 '12 at 19:56
2

Try changing the format from text to binary; this might reduce file size (and file writing time) greatly.

file.open(filename.c_str(), ios_base::binary);
...
// The following writes a vector into a file in binary format
vector<double> v;
const char* pointer = reinterpret_cast<const char*>(&v[0]);
size_t bytes = v.size() * sizeof(v[0]);
file.write(pointer, bytes);
anatolyg
  • 26,506
  • 9
  • 60
  • 134
  • How do I read from this file? ` vector v2 (32); read.open("test", ios::in | ios::binary); char byte[8]; read.read(reinterpret_cast(&v2[0]), sizeof(v2)); for(int i =0; i < 32; i++){ cout << v2[i] << endl; } ` This doenst seem to work. – pivu0 Nov 13 '12 at 16:31
  • 1
    Replace `sizeof(v2)` by `v2.size() * sizeof(double)` – anatolyg Nov 13 '12 at 17:04
  • I've reprogrammed usig binary files. It's 90% faster! Thank you! – pivu0 Nov 13 '12 at 19:55