0

In our program we are forced to call another program via popen(). The structure is basically

// external program
int main() {
    // does some stuff
    std::cout << xml << std::endl;
}

// our programm
std::string exec(const char* cmd) {
    std::array<char, 128> buffer;
    std::string result;
    std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
    
    if (!pipe)
        throw std::runtime_error("popen() failed!");

    while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr)
        result += buffer.data();
    
    return result;
}

int main() {
    const std::string xmlResult = exec("externalProgramm(.exe) --param1 --param2"); // Windows and Linux
    // do something with the result
}

From time to time the external program freezes, thus freezing our program inside fgets(). I tried to get around this problem but I cant find anything. The only solution seems to be to create a thread, wait some timeout time and if it doesnt finish in that time, .detach() it. While this solution might work for short-lived programs, ours needs to run for weeks and makes a few million calls to the external program.

Is there any way to destroy a thread or read from a popen() file without blocking everything? I know the real solution would be to "just dont call an external program that way" but we really have no other choice.

The solution needs to work on Windows and Linux. Giving my "duplicates" that only solve this on Linux does not answer my question.

Narase
  • 490
  • 2
  • 12
  • You should provide more specifics as to what the other program does, because the solution to the issue relies on that. For example, does the application return data quickly, does it return large amounts of data, etc. – l'L'l May 12 '23 at 06:54
  • @l'L'l The other program, if it doesnt freeze, returns rather quickly. Normally about 1s, sometimes up to 30s. The returned data is always <200 Bytes. The external program is not ours so I cant really tell much about whats going on internally – Narase May 12 '23 at 07:00
  • 1
    You can use the [`poll`](https://man7.org/linux/man-pages/man2/poll.2.html) function to wait for input on the pipe file descriptor. This function also allows you to specify a timeout. When `poll` reports that new input is available, you can read it using the [`read`](https://man7.org/linux/man-pages/man2/read.2.html) function. Note that this will not allow you to use the `fgets` function. You will have to program that part yourself. – Andreas Wenzel May 12 '23 at 07:10
  • @AndreasWenzel But this is Linux only, it needs to work on Windows too – Narase May 12 '23 at 07:24
  • 1
    @Narase: On Windows, the equivalent function for `poll` is [`WaitForSingleObject`](https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-waitforsingleobject). This function also allows you to specify a timeout. The equivalent function for `read` is [`ReadFile`](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-readfile). – Andreas Wenzel May 12 '23 at 07:28
  • 1
    In order to replace `fgets` as described above, you may want to derive your own [`std::streambuf`](https://en.cppreference.com/w/cpp/io/basic_streambuf) class that uses `poll`/`read` as described above, and then use [`std::istream::getline`](https://en.cppreference.com/w/cpp/io/basic_istream/getline) on your `std::streambuf` object. – Andreas Wenzel May 12 '23 at 07:32
  • I like this Idea. Never did this, but Ill figure it out. Thanks again! – Narase May 12 '23 at 07:36
  • Another option is to use your timeout, but instead of detaching your thread and leaving it hanging, do a `pipe.reset()`. This will clear the `unique_ptr` and cause the pipe to be closed with `pclose` - when the pipe closes, this should cause `fgets` to return. I do something like this in my own project, though I use `fread` instead of `fgets`. – Frodyne May 12 '23 at 08:13
  • @Frodyne I tried this solution and it doesnt work. The documentation for _pclose specifies, that it waits for the process to terminate – Narase May 12 '23 at 08:44
  • Im having a hard time understanding how I would use WaitForSingleObject and ReadFile. How do I get a HANDLE to put into the first function? Im also not sure how poll() helps in this case as it says "performs a similar task to select(2): it waits for one of a set of file descriptors to become ready". The stream is ready and sends data, it just freezes some time after that. – Narase May 12 '23 at 10:00
  • 1
    @Narase: I assume that the pipe handle can be obtained using [`CreatePipe`](https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-createpipe) or [`CreateNamedPipeA`](https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea) and then passed to `WaitForSingleObject`/`ReadFile`. – Andreas Wenzel May 12 '23 at 10:07
  • 1
    @Narase: The functions `poll` (Linux/POSIX) and `WaitForSingleObject` (Windows) allow you to wait until either of the following happens: (1) New input is available on the pipe, or: (2) The timeout expires. -- Therefore, you should only call `read`/`ReadFile` after `poll`/`WaitForSingleObject` reports that new input is available on the pipe. That way, the function call `read`/`ReadFile` will never block. Only the function call `poll`/`WaitForSingleObject` will block, and that function call allows you to set a timeout, so that it will never block indefinitely. – Andreas Wenzel May 12 '23 at 10:13

0 Answers0