Getting the output of Tcl Interpreter

Question

I am trying to get the output of Tcl Interpreter as described in answer of this question Tcl C API: redirect stdout of embedded Tcl interp to a file without affecting the whole program. Instead of writing the data to file I need to get it using pipe. I changed Tcl_OpenFileChannel to Tcl_MakeFileChannel and passed write-end of pipe to it. Then I called Tcl_Eval with some puts. No data came at read-end of the pipe.

#include <sys/wait.h>
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <tcl.h>
#include <iostream>

int main() {
    int pfd[2];
    if (pipe(pfd) == -1) { perror("pipe"); exit(EXIT_FAILURE); }
/*
        int saved_flags = fcntl(pfd[0], F_GETFL);
        fcntl(pfd[0], F_SETFL, saved_flags | O_NONBLOCK);
*/

        Tcl_Interp *interp = Tcl_CreateInterp(); 
        Tcl_Channel chan;
        int rc;
        int fd;

        /* Get the channel bound to stdout.
         * Initialize the standard channels as a byproduct
         * if this wasn't already done. */
        chan = Tcl_GetChannel(interp, "stdout", NULL);
        if (chan == NULL) {
                return TCL_ERROR;
        }

        /* Duplicate the descriptor used for stdout. */
        fd = dup(1);
        if (fd == -1) {
                perror("Failed to duplicate stdout");
                return TCL_ERROR;
        }

        /* Close stdout channel.
         * As a byproduct, this closes the FD 1, we've just cloned. */
        rc = Tcl_UnregisterChannel(interp, chan);
        if (rc != TCL_OK)
                return rc;

        /* Duplicate our saved stdout descriptor back.
         * dup() semantics are such that if it doesn't fail,
         * we get FD 1 back. */
        rc = dup(fd);
        if (rc == -1) {
                perror("Failed to reopen stdout");
                return TCL_ERROR;
        }

        /* Get rid of the cloned FD. */
        rc = close(fd);
        if (rc == -1) {
                perror("Failed to close the cloned FD");
                return TCL_ERROR;
        }

        chan = Tcl_MakeFileChannel((void*)pfd[1], TCL_WRITABLE | TCL_READABLE);
        if (chan == NULL)
                return TCL_ERROR;

        /* Since stdout channel does not exist in the interp,
         * this call will make our file channel the new stdout. */
        Tcl_RegisterChannel(interp, chan);



        rc = Tcl_Eval(interp, "puts test");
        if (rc != TCL_OK) {
                fputs("Failed to eval", stderr);
                return 2;
        }

        char buf;
        while (read(pfd[0], &buf, 1) > 0) {
            std::cout << buf;
        }

}

This looks horribly complicated. Would running some things in a subprocess be an easier approach? — Donal Fellows, May 06 '13 at 22:52

kostix · Answer 1 · 2013-05-07T16:09:40.330

I've no time at the moment to tinker with the code (might do that later) but I think this approach is flawed as I see two problems with it:

If stdout is connected to something which is not an interactive console (a call to isatty(2) is usually employed by the runtime to check for that), full buffering could be (and I think will be) engaged, so unless your call to puts in the embedded interpreter outputs so many bytes as to fill up or overflow the Tcl's channel buffer (8KiB, ISTR) and then the downstream system's buffer (see the next point), which, I think, won't be less than 4KiB (the size of a single memory page on a typical HW platform), nothing will come up at the read side.

You could test this by changing your Tcl script to flush stdout, like this:
```
puts one
flush stdout
puts two
```
You should then be able to read the four bytes output by the first puts from the pipe's read end.
A pipe is two FDs connected via a buffer (of a defined but system-dependent size). As soon as the write side (your Tcl interp) fills up that buffer, the write call which will hit the "buffer full" condition will block the writing process unless something reads from the read end to free up space in the buffer. Since the reader is the same process, such a condition has a perfect chance to deadlock since as soon as the Tcl interp is stuck trying to write to stdout, the whole process is stuck.

Now the question is: could this be made working?

The first problem might be partially fixed by turning off buffering for that channel on the Tcl side. This (supposedly) won't affect buffering provided for the pipe by the system.

The second problem is harder, and I can only think of two possibilities to fix it:

Create a pipe then fork(2) a child process ensuring its standard output stream is connected to the pipe's write end. Then embed the Tcl interpreter in that process and do nothing to the stdout stream in it as it will be implicitly connected to the child process standard output stream attached, in turn, to the pipe. You then read in your parent process from the pipe until the write side is closed.

This approach is more robust than using threads (see the next point) but it has one potential downside: if you need to somehow affect the embedded Tcl interpreter in some ways which are not known up front before the program is run (say, in response to the user's actions), you will have to set up some sort of IPC between the parent and the child processes.
Use threading and embed the Tcl interp into a separate thread: then ensure that reads from the pipe happen in another (let's call it "controlling") thread.

This approach might superficially look simpler than forking a process but then you get all the hassles related to proper synchronization common for threading. For instance, a Tcl interpreter must not be accessed directly from threads other than the one in which the interp was created. This implies not only concurrent access (which is kind of obvious by itself) but any access at all, including synchronized, because of possible TLS issues. (I'm not exactly sure this holds true, but I have a feeling this is a big can of worms.)

So, having said all that, I wonder why you seem to systematically reject suggestions to implement a custom "channel driver" for your interp and just use it to provide the implementation for the stdout channel in your interp? This would create a super-simple single-thread fully-synchronized implementation. What's wrong with this approach, really?

Also observe that if you decided to use a pipe in hope it will serve as a sort of "anonymous file", then this is wrong: a pipe assumes both sides work in parallel. And in your code you first make the Tcl interp write everything it has to write and then try to read this. This is asking for trouble, as I've described, but if this was invented just to not mess with a file, then you're just doing it wrong, and on a POSIX system the course of actions could be:

Use mkstemp() to create and open a temporary file.
Immediately delete it using the name mkstemp() returned in place of the template you passed it.

Since the file still has an open FD for it (returned by mkstemp()), it will disappear from the file system but will not be unlinked, and might be written to and read from.
Make this FD an interp's stdout. Let the interp write everything it has to.
After the interp is finished, seek() the FD back to the beginning of the file and read from it.
Close the FD when done — the space it occupied on the underlying filesystem will be reclamied.

Getting the output of Tcl Interpreter

1 Answers1