read and write file in chunks of bytes in c++

Question

I have searched for it everywhere, but I can not seem to understand how to use ios::cur. I need to read a whole file in chunks of 10 bytes, write those bytes into a buffer and then write that buffer into another file. For that I send the first 10 bytes, then the next 10 and so on. But how do i make sure the pointer starts from the last iteration's position?

char* data = 0;
int i = 0;
std::ifstream is("test.txt", std::ifstream::binary);

if (is)
{
    is.seekg(0, is.end);
    int size = is.tellg();

    cout << size << endl;

    ofstream obj;
    obj.open("new.txt");

    while (!is.eof())
    {
        for (; i <= size;)
        {
            is.seekg(i, is.beg);
            int sz = is.tellg();
            // cout<<sz<<endl;

            data = new char[sz + 1]; //  for the '\0'
            is.read(data, sz);

            data[sz] = '\0'; // set '\0' 
            cout << " data size: " << strlen(data) << "\n";
            cout << data;
            i = i + 10;
        }
        obj.close();
    }
}

Thomas Matthews · Answer 1 · 2013-11-14T19:40:25.647

You should not need to reposition the file positions.

The file positions get updated after every read operation.

You should use two file objects, one for input another for output.

In both cases, the file positions are updated after each read and write operation.

Edit 1: Simplified example

#define BUFFER_SIZE 16
unsigned char buffer[BUFFER_SIZE];
//...
while (input_file.read((char *)buffer, BUFFER_SIZE))
{
  output_file.write((char *)buffer, BUFFER_SIZE);
}

If the input_file is position at offset 0, then after the first read, the file position will be at 16. This can be verified by:

int read_file_position = input_file.tellg();  
cout << "input file position: " << read_file_position << endl;
while (input_file.read((char *)buffer, BUFFER_SIZE))
{
  read_file_position = input_file.tellg();
  cout << "input file position: " << read_file_position << endl;
  output_file.write((char *)buffer, BUFFER_SIZE);
}

I have two objects, "is" for reading, obj for writing. Can you please elaborate "You should not need to reposition the file positions. The file positions get updated after every read operation."? In the code Ive pasted, the first 10 bytes do not get printed. I dont understand the reason behind this. :( — Basilisk, Nov 14 '13 at 18:48

Ashalynd · Answer 2 · 2013-11-14T21:57:14.097

1

I think it can be simplified to that:

char* data = 0;
int i = 0;
int size = 0;
int sz = 10;
std::ifstream is("test.txt", std::ifstream::binary);
std::ofstream obj;
data = new char[sz+1];
int read_sz = 0;

if (is)
{
    obj.open("new.txt");
    read_sz = is.tellg(); //we are at the beginning of the file now
    while (!is.eof())
    {
        int old_read_sz = read_sz; // old position - before next read
        is.read(data, sz);
        read_sz = is.tellg(); // new position, we've read read_sz-old_read_sz bytes
        data[read_sz-old_read_sz] = '\0'; // set '\0'
        // I hope you wanted the two lines below only for debugging purposes :)
        cout << " data size so far: " << read_sz << "\n";
        cout << data << endl;
        obj.write(data, read_sz-old_read_sz);
     }
    obj.close();
    is.close();
    cout << "total data size: "<< read_sz << endl;
}

Once you have read a data chunk, your file cursor has already moved past that chunk, so you do not need to reposition it yourself (and moving from the current position to the end / beginning and back can be costly - timewise - especially if you have a large input file).

By the way, here is a tutorial about the topic: http://www.cplusplus.com/doc/tutorial/files/

Update: forgot that is.read() != old C-style read :-}

edited Nov 14 '13 at 21:57

answered Nov 14 '13 at 18:51

Ashalynd

12,363
2
34
37

data = new char[sz+1]; // for the zero to generate debugging output (actually std::string(data, read_sz) would be better) – Nov 14 '13 at 19:04
oh indeed, that was the intention - fixed :) – Ashalynd Nov 14 '13 at 19:05
Oh wow. Yes that was for debugging. However, I'm getting this error: "cannot convert from 'class std::basic_istream >' to 'int' " at line "int read_sz = is.read(data, sz);". Could you please tell me what is read_sz doing? – Basilisk Nov 14 '13 at 19:18
Replace the "int read_sz = ..." with "...; int read_sz = is.gcount();" – Thomas Matthews Nov 14 '13 at 19:27
Omg, its working now! Thank you :D But everywhere that I read about sending data byte to byte included seekg and tellg() calls. Are they recommended in c++? – Basilisk Nov 14 '13 at 19:34
Also, I am making packets that have to be sent over udp. So for every packet I send (containing data of 10 bytes each), I will need to keep the writing-to-file part out of the while loop, right? – Basilisk Nov 14 '13 at 19:40
Ah sorry, I forgot that read for a stream doesn't return the number of read bytes (like C read used to do). – Ashalynd Nov 14 '13 at 21:49
Have changed the code a bit. Regarding `seekg` / `tellg` : `tellg` gives you your current position (which is strictly speaking of type `streampos` but it can be converted to `int`). `seekg` is only needed when you have to relocate your file cursor (for example, if you are doing non-sequential reads). – Ashalynd Nov 14 '13 at 21:59

read and write file in chunks of bytes in c++

2 Answers2