Undefined characters while reading the file content without new line at the end of file

Question

File (settings.txt) to parse:

FULLSCREEN=On
V_SYNC=On [no "\n" at the end of file]

In case with no ENTER "\n" output is:

MapKey= FULLSCREEN      MapValue= On
MapKey= V_SYNC  MapValue= Onřřřř

With ENTER "\n" at the end of file output is correct (without "řřřř"):

MapKey= FULLSCREEN      MapValue= On
MapKey= V_SYNC  MapValue= On

How to change the program to work without adding new line at the end of file? Code:

#include<iostream>
#include<fstream>
#include<sstream>
#include<vector>
#include<cstdint>
#include<memory>


int main()
{
    std::vector<std::pair<std::string, std::string>> container;
    std::ifstream containerFile("settings.txt", std::ifstream::binary);
    containerFile.seekg(0, containerFile.end);
    std::uint64_t fileSize = containerFile.tellg();
    containerFile.seekg(0);
    std::unique_ptr<char> fileBuffer(new char[fileSize]);
    containerFile.read(fileBuffer.get(), fileSize);

    std::istringstream fileContent(fileBuffer.get());
    std::string fileLine;
    while (std::getline(fileContent, fileLine))
    {
        std::istringstream bufferLine(fileLine);
        std::string option;
        if (std::getline(bufferLine, option, '='))
        {
            std::string value;
            if (std::getline(bufferLine, value))
            {
                container.emplace_back(make_pair(option, value));
            }
        }
    }
    for (auto &element : container)
    {
        std::cout << "MapKey= " << element.first << "   MapValue= " << element.second << std::endl;
    }
    containerFile.close();
}

`std::istringstream fileContent(fileBuffer.get());` - an improperly null-terminated char buffer is converted to a std::string. There are many ways to fix this, like explicitly construct the string with a length: `std::istringstream fileContent(std::string(fileBuffer.get(), fileSize));` — Christopher Oicles, Jan 10 '17 at 01:33
And if this is a Windows system, and if this is a standard text file, reading with `std::ifstream::binary` will keep the extra carriage returns before each linefeed character : (`"\r\n"`), which might also cause parsing problems. — Christopher Oicles, Jan 10 '17 at 01:39

arturx64 · Answer 1 · 2017-01-10T02:29:46.610

You can rewrite your code in this way:

std::vector<std::pair<std::string, std::string>> container;
std::ifstream containerFile("settings.txt");
containerFile.seekg(0, containerFile.end );
std::uint64_t fileSize = containerFile.tellg();
containerFile.seekg(0);

/* Changed code. Begin*/
containerFile >> std::noskipws;
std::vector<char> buffer;
buffer.reserve(fileSize);
std::copy(std::istream_iterator<char>(containerFile), std::istream_iterator<char>(), std::back_inserter(buffer));
buffer.push_back('\0');
std::istringstream fileContent(&buffer.front());
/* Changed code. End*/

std::string fileLine;
while (std::getline(fileContent, fileLine))
{
    std::istringstream bufferLine(fileLine);
    std::string option;
    if (std::getline(bufferLine, option, '='))
    {
        std::string value;
        if (std::getline(bufferLine, value))
        {
            container.emplace_back(make_pair(option, value));
        }
    }
}
for (auto &element : container)
{
    std::cout << "MapKey= " << element.first << "   MapValue= " << element.second << std::endl;
}
containerFile.close();

First of all you have to use the next flag: containerFile >> std::noskipws; It allows you to use whitespaces. Tab spaces, carriage returns and blank spaces are all considered whitespaces according to documentation.

Correct string representation requires Null character in the end, so the next line buffer.push_back('\0'); adds '\0' to the end of the buffer.

I don't see why '\0' needs adding at all. If this was a ASCIIZ C-string, yes. But for a C++ string it is not required. — SJHowe, Jan 10 '17 at 12:07

Undefined characters while reading the file content without new line at the end of file

1 Answers1