What is the best way (performance driven) to convert and write variables to a file in c++?

Question

I want to write something like this to a file:

0x050 addik r13, r0, 4496
0x054 addik r2, r0, 2224
0x058 addik r1, r0, 7536
0x05c brlid r15, 200
...

And so on... Its a program instruction trace which will have thousands of lines.

I am reading from an 'elf' decoding the instruction, creating an object, setting its address, instruction, name and registers parameters and then writing it in the above format to a file.

What it is the best way, measuring in speed/performance, to do this?

Now I have this (still just hexadecimals) and I don't know if it is the best way to continue writing my code:

Converting function:

static std::string toHex(const T &i) {
    std::stringstream stream;
    stream << "0x" 
           << std::setfill ('0') << std::setw(sizeof(T)*2) 
           << std::hex << i;
    return stream.str();
};

And the writing:

while((newInstruction = manager->newInstruction())){
    stream  << Utils::toHex(newInstruction->getAddress())
            << " "
            << Utils::toHex(newInstruction->getInstruction())
            << endl;
    trace->writeFile(stream.str());
    stream.str(std::string());
}

EDIT:

So I have reached a faster solution based on the answers.

For one I implemented the solution given by Escualo to stop creating objects each time I read a new instruction.

And then I read the answer given by Thomas Matthews and gave me the idea to not write to my file at every instruction read, so the stringstream now works like a buffer with size 1024, and when it surpasses that value then writes the stream to the file:

while((newInstruction = manager->newInstruction())){
    stream  << myHex<unsigned int> << newInstruction->getAddress() << ' '
            << myHex<uint32_t> << newInstruction->getInstruction();
    if(stream.tellp() > 1024){
        trace->writeFile(stream.str());
        stream.str(std::string());
    }
}

What makes speed/performance the most important aspect of this code (rather than readability, maintainability, reusability, or other nice things)? I ask because when speed _really_ is important, the fastest way to write variables to a file is to directly copy the bytes (which are not human readable) and run a slower program later, when time is no longer the most important thing, to translate them to human-readable text. — David K, Jul 10 '15 at 17:52
The issue I have is that it will have to manage assembly instructions from another programs, which can reach easily one million instructions, and I have to decode every instruction which takes time, then detect sequences which takes even more time and then write everything to files. — joaomlap, Jul 10 '15 at 17:56
Which part of your code has your profiling identified as the bottleneck, that just isn't fast enough? — IInspectable, Jul 10 '15 at 18:00
`void File::writeFile(const string &s){ if(file.is_open()) file << s << endl; else return; }` — joaomlap, Jul 10 '15 at 18:01
My program in C is doing the same plus the decoding every instruction and detect patterns in the instruction trace and it is taking less time to do it. I am concerned that when it is fully coded in C++ will take a large amount of time to do everything. — joaomlap, Jul 10 '15 at 18:03
I think the bottleneck is using stringstreams and then writing it to a file. — joaomlap, Jul 10 '15 at 18:07
Profiling relieves you of the erroneous task of *guessing, where the bottleneck is*. — IInspectable, Jul 10 '15 at 18:14
For scalability. I need to add more functionality now and it will grow even bigger after I stop working on it, because other people will work whith it too, so I took the decision to recode it in C++. — joaomlap, Jul 10 '15 at 18:22

score 1 · Answer 1 · answered Jul 10 '15 at 19:15

Since file I/O is slower than the time for formatting, I suggest formatting into a buffer, the block writing the buffer to the file.

char text_buffer[1024];
unsigned int bytes_formatted = 0;
for (unsigned int i = 0; i < 10; ++i)
{
  int chars_formatted = snprintf(&text_buffer[bytes_formatted],
                                 1024-bytes_formatted,
                                 "hello %d", i);
  if (chars_formatted > 0)
  {
    bytes_formatted += chars_formatted;
  }
}
my_file.write(text_buffer, bytes_formatted);

Most file I/O operations have a constant overhead regardless of the operation. So the more data that can be written during one operation, the better.

score 0 · Accepted Answer · answered Jul 10 '15 at 18:40

For one, I would avoid creating and destroying an std::stringstream for every call to your formatting function.

Recall that the I/O manipulators are nothing but functions which return the stream itself. For example, a manipulator doing exactly what you indicated above, but without resorting to a temporary std::stringstream could look like:

#include <iostream>
#include <iomanip>

template<typename T,
         typename CharT,
         typename Traits = std::char_traits<CharT> >
inline std::basic_ostream<CharT, Traits>&
myhex(std::basic_ostream<CharT, Traits>& os) {
  return os << "0x"
            << std::setfill('0')
            << std::setw(2 * sizeof(T))
            << std::hex;
}

int main() {
  int x;
  std::cout << myhex<int> << &x << std::endl;
}

to print (for example):

0x0x7fff5926cf9c

To clarify: I do not know why you choose the fill, width, prefix, and format; I am just showing you how to create an I/O manipulator that does not entail creating and destroying temporary objects.

Notice that the manipulator will work with any std::basic_ostream<CharT, Traits> such as std::cout, std::cerr, std::ofstream, and std::stringstream.

I liked the idea of not creating a disposable object everytime I read an instruction, but I am struggling with the solution you presented since I am getting this compiling error: `main.cpp:23: error: default template arguments may not be used in function templates without -std=c++0x or -std=gnu++0x make: *** [main.o] Error 1 ` — joaomlap, Jul 10 '15 at 19:13
So, just do what the compiler tells you: add `-std=c++0x` to your compilation line (this tells the compiler to use the C++11 standard). — Escualo, Jul 10 '15 at 19:39

What is the best way (performance driven) to convert and write variables to a file in c++?

2 Answers2