How can I read a line from a stringstream only if it contains any newline?

Question

I'm reading some network data into a stringstream as an input_buffer.

The data is ASCII lines separated by a LF char.

The input_buffer may be in a state where there is only a partial line in it.

I'm trying to call getline (), but only when there actually is a new newline char in the stringstream. In other words it should extract completed lines, but leave a partial line in the buffer.

Here is a MVCE:

#include <string>
#include <sstream>
#include <iostream>

int
main (void)
{
  std::stringstream input_buffer;
  input_buffer << "test123\nOK\n";
  while (input_buffer.str ().find ('\n') != std::string::npos)
    {
      std::string line;
      std::getline (input_buffer, line, '\n');
      std::cout << "input_buffer.str ().size: " << input_buffer.str ().size () << "\n";
      std::cout << "line: " << line << "\n";
    }
  return 0;
}

It currently does not terminate, here is a fragment of the output:

input_buffer.str ().size: 11
line: test123
input_buffer.str ().size: 11
line: OK
input_buffer.str ().size: 11
line: 
input_buffer.str ().size: 11
...

How can I read a line from a stringstream only if it contains any newline?

Edit: For clarification here is another code sample with partial input:

#include <string>
#include <sstream>
#include <iostream>
#include <vector>

void
extract_complete_lines_1 (std::stringstream &input_buffer, std::vector<std::string> &lines)
{
  while (input_buffer.str ().find ('\n') != std::string::npos)
    {
      std::string line;
      std::getline (input_buffer, line, '\n');
      lines.push_back (line);
    }
}

void
print_lines (const std::vector<std::string> &v)
{
  for (auto l : v)
    {
      std::cout << l << '\n';
    }
}

int
main (void)
{
  std::vector<std::string> lines;
  std::stringstream input_buffer {"test123\nOK\npartial line"};
  extract_complete_lines_1 (input_buffer, lines);
  print_lines (lines);
  return 0;
}

This should print "test123" and "OK", but not "partial line".

It doesn't terminate because you're repeatedly searching from the beginning of the string. Reading from `input_buffer` doesn't alter the underlying contents. — molbdnilo, Jan 07 '17 at 13:54
When you have hit the end, `input_buffer.eof()` will be true. — Bo Persson, Jan 07 '17 at 14:04
[This](http://stackoverflow.com/q/12351163/1460794) might be relevant. — wally, Jan 07 '17 at 14:49
I'm trying to avoid blocking, the whole network code is implemented with `libuv` and the `uvw` library. I get a char* every time there is data available, but it might only be partial data. I then want to read the part of the data that is complete and give control back to the event loop. When the next data is available the code will be called again. The stringstream is the only state involved, it serves as a FIFO buffer. — Promi, Jan 07 '17 at 14:52
So you have some underlying network functions (not stream based) and you want to start working with the data coming in as a stream. But the problem is that stringstreams don't know if they're open or closed, so `getline` will return even without a newline if it gets to `eof` (which is not what is required here as more will be added to the stream later). — wally, Jan 07 '17 at 14:53
The actual code is here: https://gist.github.com/promi/659fd1cf585238863667c0ed3709d565 (which is only test code so far, so the line processing is not complete). — Promi, Jan 07 '17 at 14:59

score 0 · Answer 1 · edited May 23 '17 at 12:33

As mentioned here, you could override the underflow function of the buffer so that it will refill using a function that you can specify.

Here is an example adapted from here:

#include <iostream>
#include <sstream>
#include <string>

class Mybuf : public std::streambuf {
    std::string line{};
    char ch{}; // single-byte buffer
protected:
    int underflow() override {
        if(line.empty()) {
            std::cout << "Please enter a line of text for the stream: ";
            getline(std::cin, line);
            line.push_back('\n');
        }
        ch = line[0];
        line.erase(0, 1);
        setg(&ch, &ch, &ch + 1); // make one read position available
        return ch; 
    }
public:
    Mybuf(std::string line) : line{line} {};
};

class mystream : public std::istringstream {
    Mybuf mybuf;

public:
    mystream(std::string line) : std::istringstream{}, mybuf{line}
    {
        static_cast<std::istream&>(*this).rdbuf(&mybuf);
    }
};

int main()
{
    mystream ms{"The first line.\nThe second line.\nA partial line"};
    for(std::string line{}; std::getline(ms, line); )
        std::cout << "line: " << line << "\n";
}

Output:

line: The first line.
line: The second line.
Please enter a line of text for the stream: Here is more!
line: A partial lineHere is more!
Please enter a line of text for the stream:

Unfortunately the connection is long living, I get chunks of the data in a callback, so I can't rely on a "proper" istream. That's why I'm adding the data to a stringstream. Otherwise `getline ()` wouldn't work at all. — Promi, Jan 07 '17 at 14:22
Ah ok. So you create a `stringstream` from the network functions. Then you want the stringstream to remain open after each line is read and you don't want to read a line up to `eof`, only up to newlines? — wally, Jan 07 '17 at 14:42
Yes, the stringstream stays open until the connection is closed. It serves as a buffer to recombine partial data into whole lines. Partial data should not be read from it, until it is completed (But then it's not partial anymore ;) ). It serves as a FIFO. — Promi, Jan 07 '17 at 14:48

Promi · Accepted Answer · 2017-01-07T16:37:31.690

I think that it's not easily possible with std::stringstream. I tried to manipulate the stream position with tellg () and seekg (), but they don't behave like I expected.

I have found a solution using a std::vector<char> as a buffer:

#include <string>
#include <sstream>
#include <iostream>
#include <vector>
#include <iterator>
#include <algorithm>

void
extract_complete_lines (std::vector<char> &buf, std::vector<std::string> &lines)
{
  auto pos = std::end (buf);
  while ((pos = std::find (std::begin (buf), std::end (buf), '\n')) != std::end (buf))
    {
      std::string line (std::begin (buf), pos);
      buf.erase (std::begin(buf), pos + 1);
      lines.push_back (line);
    }
}

void
print_lines (const std::vector<std::string> &v)
{
  for (auto l : v)
    {
      std::cout << l << '\n';
    }
}

int
main (void)
{
  std::vector<std::string> lines;
  const std::string test_input = "test123\nOK\npartial line";
  std::vector<char> input_buffer {std::begin (test_input), std::end (test_input)};
  extract_complete_lines_1 (input_buffer, lines);
  print_lines (lines);
  return 0;
}

It prints the first two lines as expected and the "partial line" is left in the vector.

Or even better, a std::vector<char> is not too different from a std::string:

#include <string>
#include <sstream>
#include <iostream>
#include <vector>
#include <iterator>
#include <algorithm>

void
extract_complete_lines (std::string &buf, std::vector<std::string> &lines)
{
  std::string::size_type pos;
  while ((pos = buf.find ('\n')) != std::string::npos)
    {
      lines.push_back (buf.substr (0, pos));
      buf.erase (0, pos + 1);
    }
}

void
print_lines (const std::vector<std::string> &v)
{
  for (auto l : v)
    {
      std::cout << l << '\n';
    }
}

int
main (void)
{
  std::vector<std::string> lines;
  std::string input_buffer = "test123\nOK\npartial line";
  extract_complete_lines (input_buffer, lines);
  print_lines (lines);
  return 0;
}

score 0 · Answer 3 · answered May 09 '23 at 12:13

Why not just look for the newlines?

#include <array>
#include <iostream>
#include <sstream>
#include <unistd.h>


using namespace std;
int main() {
  array<char, 128> buffer {};
  std::string line;
  stringstream result;
  auto pipe = ::fopen("/home/tom/CLionProjects/test/data.txt", "r");

  auto logString = [](const std::string & line)
  {
    std::cout << "line: " << line << std::endl;
  };

  auto reported = 0u;
  while (fgets(buffer.data(), buffer.size(), pipe) != nullptr) {
    result << std::string(buffer.data());
    if (result.str().find('\n', reported) != std::string::npos && std::getline(result, line))
    {
      logString(line);
      reported += line.size() + 1; // +1 for the \n
    }
  }
  if (std::getline(result, line)) {
    logString(line);
  }
}

How can I read a line from a stringstream only if it contains any newline?

3 Answers3