How to capture stdout output but also show progress

Question

I have a function named print() that print numbers every 2 seconds, this function runs in a goroutine.

I need to pass its stdout printing to a variable and print it, but not one time, until it finish.
I need to have a some scanner in an infinite loop that will scan for the stdout and print it, once the function done the scanner will done too.

I tried to use this answer but it doesn't print anything.
This is what I tried to do:

package main

import (
    "bufio"
    "fmt"
    "os"
    "sync"
    "time"
)


func print() {

    for i := 0; i < 50; i++ {
        time.Sleep(2 * time.Second)
        fmt.Printf("hello number: %d\n", i)
    }
}

func main() {
    old := os.Stdout // keep backup of the real stdout

    defer func() { os.Stdout = old }()
    r, w, _ := os.Pipe()
    os.Stdout = w

    go print()


    var wg sync.WaitGroup

    c := make(chan struct{})
    wg.Add(1)


    defer wg.Done()
    for {
        <-c
        scanner := bufio.NewScanner(r)
        for scanner.Scan() {
            m := scanner.Text()
            fmt.Println("output: " + m)
        }

    }

    c <- struct{}{}

    wg.Wait()
    fmt.Println("DONE")

}

I also tried to use io.Copy to read the buffer like that but it didn't work too:

package main

import (
    "bytes"
    "fmt"
    "io"
    "os"
    "time"
)


func print() {

    for i := 0; i < 50; i++ {
        time.Sleep(2 * time.Second)
        fmt.Printf("hello number: %d\n", i)
    }
}

// https://blog.kowalczyk.info/article/wOYk/advanced-command-execution-in-go-with-osexec.html
func main() {
    old := os.Stdout // keep backup of the real stdout

    defer func() { os.Stdout = old }()
    r, w, _ := os.Pipe()
    os.Stdout = w

    go print()

    fmt.Println("DONE 1")
    outC := make(chan string)

    for {

        var buf bytes.Buffer
        io.Copy(&buf, r)
        outC <- buf.String()

        out := <-outC
        fmt.Println("output: " + out)
    }

    // back to normal state
    w.Close()


    fmt.Println("DONE")

}

I have no clue what you're trying to do, but assigning os.Stdout to something else is wrong. If you want to print somewhere else than standard out use [Fprintln](https://golang.org/pkg/fmt/#Fprintln) instead of Println. — Peter, Aug 24 '20 at 15:14
Your first snippet has few fatal problems. E.g. you try to capture `fmt.Printf` using `Pipe` and `Scanner` - this could potentially work, but then you send captured result to `fmt.Println()` which also prints to `Stdout` and Stdout is already redirected to `Pipe()`, so result is not printed to console (as you perhaps expect), instead it goes to `Scanner` again . There are other issues but this one should be solved first. — maxim_ge, Aug 24 '20 at 15:29
@maxim_ge I changed the `fmt.Println` to `fmt.Printf`, it still doesn't work. I tried to change `fmt.Printf` to `fmt.Println` and it also doesn't work :( — E235, Aug 24 '20 at 15:33
@Peter I changed `fmt.Println` to `fmt.Fprintf(w, "output: " + m)` but it still doesn't print anything. But I need to read it from the stdout of the pipe, this is the problem. And basically what I am trying to do is to read the printing of function while it still in progress. — E235, Aug 24 '20 at 15:35
@E235 I'd refactor your code like this https://play.golang.org/p/MfAGaC52o_P — maxim_ge, Aug 24 '20 at 17:37
If I may ask, what is the purpose of this? I am curious. Can you just send data from goroutine to main loop via chan directly instead of trying to print and read? And in second snippet, what is the purpose of sending string to `outC` chan and reading it one line after from the same chan? Can you just use `buf.String()` inside println like this `fmt.Println("output: ", buf.String())`? — Aleksandar, Aug 24 '20 at 22:22
@maxim_ge yes it works, the only problem that the `print()` function is like a blackbox so I don't want to modify it. @Aleksandar - the general purpose is to capture stdout when I am using the Go interpreter ("yaegi") in an agent client and I need to capture the data and send to the server. The agent runs golang code that takes 2-5 minutes, in the meanwhile I want it to send the output to the server. For simplicity I created this example which if you assume `print()` is like a blackbox, it should be enough. — E235, Aug 25 '20 at 08:00

score 1 · Accepted Answer · answered Aug 25 '20 at 08:54

It is possible to run print() as a "blackbox" and capture its output though it is a little bit tricky and does not work on Go Playground.

package main

import (
    "bufio"
    "fmt"
    "os"
    "runtime"
    "time"
)


func print() {
    for i := 0; i < 50; i++ {
        time.Sleep(100 * time.Millisecond)
        fmt.Printf("hello number: %d\n", i)
    }
}

func main() {

    var ttyName string
    if runtime.GOOS == "windows" {
    fmt.Println("*** Using `con`")
        ttyName = "con"
    } else {
    fmt.Println("*** Using `/dev/tty`")
        ttyName = "/dev/tty"
    }   

    f, err := os.OpenFile(ttyName, os.O_WRONLY, 0644)
    if err != nil {
        panic(err)
    }
    defer f.Close()

    r, w, _ := os.Pipe()
    oldStdout := os.Stdout
    os.Stdout = w
    defer func() { 
        os.Stdout = oldStdout
        fmt.Println("*** DONE")
    }()

    fmt.Fprintln(f, "*** Stdout redirected")

    go func(){
       print()
       w.Close()
       r.Close()          
    }()

    c := make(chan struct{})
    go func(){c <- struct{}{}}()
    defer close(c)

    <-c
    scanner := bufio.NewScanner(r)
    for scanner.Scan() {
        m := scanner.Text()
        fmt.Fprintln(f, "output: " + m)
    }
}

Great! It works :) thanks! Thanks for understanding the importance of using the `print()` as blackbox, it was important for me. FYI, it doesn't work on Windows, only on Linux, but for now it enough for me. The next thing I need to do is to see how to capture the printing when the `print()` already finished but I will try by myself and if I will have a problem I will open a new question — E235, Aug 25 '20 at 11:02
I noticed that when I remove the `time.Sleep(100 * time.Millisecond)` it doesn't print it, is it possible to make the scanner to wait untill the `print()` funcion finish? — E235, Sep 01 '20 at 06:57
@E235 in fact you have to wait until scanner finishes with reading. Best way would be to flush Stdout somehow but I do not know how to do that. As a work around you can add a call to `time.Sleep()` immediately after `print()`. This will give scanner some time — maxim_ge, Sep 01 '20 at 08:15

How to capture stdout output but also show progress

1 Answers1