Reading from file into vector line by line works incorrect for binary data C++

Question

There is a template to fill vector <T> from the file:

template<typename T, typename A>
void fill_vector_from_file(const std::string  &filePath, std::vector<T, A> & target)
{
    std::ifstream is(filePath, std::ifstream::in);

    is.seekg(0, std::ifstream::end);
    std::size_t size = is.tellg();
    is.seekg(0, std::ifstream::beg);
    target.reserve(size);

std::string line;
while (std::getline(is, line))
{
    std::istringstream line_in(line);
    while (line_in)
    {
        T val = 0;
        if (line_in >> val)
        {
            target.push_back(val);
        }
    }
}
is.close();

Data in files can be int or binary and stored one number per line , for example:
For int:

2 
-3
4

and for binary:

010
111
001

When I checked template with std::vector<int> v1 for integers and std::vector<unsigned char> v2 , result of v2[0] was 0 instead of 010.
(I supposed, that I should use unsigned char for to store binary)

Question: Is there any way to modify template , so the result of v2[0] will be as expected (010).

`unsigned char` is only 1 byte in size. You are reading the file as a text file, line by line. Reading a line like `"2"` as an `int` works fine, of course, since `operator>>` supports that conversion. But you can't read in a line like `"010"` as an `unsigned char`. The STL has no I/O manipulator for reading binary-digit strings. The code shown is going to read the individual chars, so it will read `'0'`, `'1'`, `'0'` instead. You will have to add extra code logic to read the `"010"` line and apply your own conversion logic to turn it into an `unsigned char` (0x02). — Remy Lebeau, Feb 28 '18 at 19:18
You have to read second file knowing that data is stored as string which represents binary. Currently it doesn't seem like you have this kind of detection in your template. — Killzone Kid, Feb 28 '18 at 19:21
_>>I supposed, that I should use unsigned char for to store binary_ You can try `std::bitset<3>` for type `T` and then convert it to `unsigned char` later — Killzone Kid, Feb 28 '18 at 19:25

S. Huber · Answer 1 · 2018-02-28T19:59:08.253

There is a lot wrong with this code, but your original question has the following answer:

Converting a 0-1 ASCII string to integer. Your code line_in >> val for val being of type unsigned char reads a single character, like '0' and '1'. You want to convert an ASCII string made of '0's and '1's, which forms a base-2 number, into an integer. In this SO answer you find that val = std::stoi(line, nullptr, 2); does that for you.

So, neither "-3" nor "101" are integers but strings that represent integers in base-10 and base-2, and stoi() does the conversion to integers for you. In case of base-10, however, the iostream operator>>() works as well. (You may also have a look into std::setbase().)

score 1 · Accepted Answer · answered Feb 28 '18 at 20:20

The 2nd file seems to contain strings of binary format. Let's assume they are always 3 bits long, in which case if you use std::bitset<3> you will read each number in full. If you use unsigned char then you will only read one digit at a time. Here is your function, slightly modified with examples of reading different files (format of which you know before hand I suppose). As a bonus there is also example of how to convert std::bitset vector into unsigned char should you need it.

#include <vector>
#include <iostream>
#include <string>
#include <fstream>
#include <bitset>
#include <algorithm> // std::transform

template<typename T, typename A>
void fill_vector_from_file(std::string const &filePath, std::vector<T, A> &vec)
{
    std::ifstream ifs(filePath);
    T val;

    while (ifs >> val)
        vec.push_back(val);
}

int main()
{
    // make sample files
    std::ofstream ofs("myfile.txt");
    ofs << "2\n" << "-3\n" << "4\n";
    ofs.close();
    ofs.open("myfile2.txt");
    ofs << "010\n" << "111\n" << "001\n";
    ofs.close();


    // fill <int> vector
    std::vector<int> vi;
    fill_vector_from_file("myfile.txt", vi);
    // print int vector
    for (auto n : vi)
        std::cout << n << std::endl;


    // fill <bitset> vector 
    std::vector<std::bitset<3>> vbs;
    fill_vector_from_file("myfile2.txt", vbs);
    // print bitset vector
    for (auto n : vbs)
        std::cout << n << std::endl;


    // [OPTIONAL] convert to vector <unsigned char>
    std::vector<unsigned char> vuc(vbs.size());
    std::transform(vbs.begin(), vbs.end(), vuc.begin(),
        [](std::bitset<3> const &bs) -> unsigned char { return static_cast<unsigned char>(bs.to_ulong()); });
    // print vector <unsigned char>
    for (auto n : vuc)
        std::cout << int(n) << std::endl;


    return 0;
}

Working demo: http://coliru.stacked-crooked.com/view?id=42aa04e34e4194c1

Reading from file into vector line by line works incorrect for binary data C++

2 Answers2