Running command and get the output online

Question

I use the following code which works, The problem is that the output is printed just as the process finished to execute, I want to print to the screen the output live and not provide all the output when the process finishes, how can I achieve this?

cmdParams := [][]string{
    {filepath.Join(dir,path), "npm", "install"},
    {filepath.Join(pdir,n.path), "gulp"},
}

for _, cp := range cmdParams {
    log.Printf("Starting %s in folder %s...", cp[1:], cp[0])
    cmd := exec.Command(cp[1], cp[2:]...)
    cmd.Dir = cp[0]
    // Wait to finish, get output:
    out, err := cmd.Output()
    if err != nil {
        log.Printf("Error running %s: %v\n", cp[1:], err)
        return
    }
    log.Println("Finished %s, output: %s", cp[1:], out)
}

update when trying the proposed solution I got the output like

2018/02/18 11:11:57  Starting [npm install] in folder ...
2018/02/18 11:12:14 adde
2018/02/18 11:12:14 d 56
2018/02/18 11:12:14 3 pa
2018/02/18 11:12:14 ckag
2018/02/18 11:12:14 es i
2018/02/18 11:12:14 n 15
2018/02/18 11:12:14 .477
2018/02/18 11:12:14 s
2018/02/18 11:12:14 Finished %s [npm install]

Possible duplicate of [Streaming commands output progress](https://stackoverflow.com/questions/30725751/streaming-commands-output-progress/30726590#30726590). Also see related: [How to get the realtime output for a shell command in golang?](https://stackoverflow.com/questions/37091316/how-to-get-the-realtime-output-for-a-shell-command-in-golang/37137583#37137583) — icza, Feb 18 '18 at 08:03
@icza - so I should use `oneByte := make([]byte, 1) for { _, err := stdout.Read(oneByte) if err != nil { break } fmt.Printf("%c", oneByte[0]) }` for this case ? if so where ? it's a bit confusing ... — , Feb 18 '18 at 08:13
Use the solution from the accepted answer of the suggested duplicate. — icza, Feb 18 '18 at 08:19
@icza - do you mean -`scanner := bufio.NewScanner(stdout) scanner.Split(bufio.ScanRunes)` ? — , Feb 18 '18 at 08:20
@icza - it will be great if you can provide example with my context, since I read the post and it's difference , I run several command and I use the `cmd.Output()` which I dont understand how to relate them... — , Feb 18 '18 at 08:22
See my answer below. I haven't tested it, but hopefully it works right away. — icza, Feb 18 '18 at 08:40

icza · Accepted Answer · 2018-02-18T20:13:33.193

3

Using the solution presented in this answer: Streaming commands output progress

cmdParams := [][]string{
    {filepath.Join(dir, path), "npm", "install"},
    {filepath.Join(pdir, n.path), "gulp"},
}
for _, cp := range cmdParams {
    log.Printf("Starting %s in folder %s...", cp[1:], cp[0])
    cmd := exec.Command(cp[1], cp[2:]...)
    cmd.Dir = cp[0]

    stdout, err := cmd.StdoutPipe()
    if err != nil {
        log.Printf("%s cmd.StdoutPipe() error: %v\n", cp[1:], err)
        return
    }
    // Start command:
    if err = cmd.Start(); err != nil {
        log.Printf("%s start error: %v\n", cp[1:], err)
        return
    }

    // Stream command output:
    scanner := bufio.NewScanner(stdout)
    scanner.Split(bufio.ScanRunes)
    for scanner.Scan() {
        fmt.Print(scanner.Text())
    }
    if scanner.Err() != nil {
        log.Printf("Reading %s stdout error: %v\n", cp[1:], err)
        return
    }

    // Get execution success or failure:
    if err = cmd.Wait(); err != nil {
        log.Printf("Error running %s: %v\n", cp[1:], err)
        return
    }
    log.Printf("Finished %s", cp[1:])
}

Some explanation:

This line:

scanner := bufio.NewScanner(stdout)

Creates a bufio.Scanner that will read from the pipe that supplies the output written by the process.

This line:

scanner.Split(bufio.ScanRunes)

Instructs the scanner to read the input by runes instead of the default by-lines.

Note that the above example only streams the standard output of the process. If you also need its standard error stream, you could also read from Command.StderrPipe().

Also note that this won't work with some commands that don't write everything to their standard output or error. For details, see How to get the realtime output for a shell command in golang?

edited Feb 18 '18 at 20:13

answered Feb 18 '18 at 08:35

icza

389,944
63
907
827

Thanks 1+ , can you see my updates why the output is like this ? – Feb 18 '18 at 09:15
I think it's becouse of `count, err := stdout.Read(oneRune)` , any idea how to overcome this? – Feb 18 '18 at 09:18
Thanks, I've tried it and now I got even worse `2018/02/18 11:44:14 a 2018/02/18 11:44:14 d 2018/02/18 11:44:14 d 2018/02/18 11:44:14 e 2018/02/18 11:44:14 d 2018/02/18 11:44:14 2018/02/18 11:44:14 5` – Feb 18 '18 at 09:45
each byte in a new line – Feb 18 '18 at 09:45
@RaynD Do note when streaming the output I use `fmt.Printf()` and **not** `log.Printf()`. – icza Feb 18 '18 at 09:46
Ohh now I see, there is no way to use log? and why it works on fmt and not log? – Feb 18 '18 at 09:48
@RaynD If it works with `fmt`, can you also try the previous version with `oneRune := make([]byte, utf8.UTFMax)`? – icza Feb 18 '18 at 09:49
Sure not checking it – Feb 18 '18 at 09:49
@RaynD And it works with `fmt` because `fmt.Printf()` does not append a newline to the end, while `log.Printf()` does append a newline. – icza Feb 18 '18 at 09:49
Thanks a lot, I missed the `fmt` part , it works and Im closing the question , it will be great if you can provide additaionl info about this 3 lines ,` `scanner := bufio.NewScanner(stdout) scanner.Split(bufio.ScanRunes) oneRune := make([]byte, utf8.UTFMax)` what is utf.UTFMax` etc – Feb 18 '18 at 09:56
Thanks a lot sir, one last question , assume I need to execute a code when the all process has finished (all the loops is done) where is the best to do it ? – Feb 18 '18 at 10:13
@RaynD Edited the answer to add more explanation. If you want to run a process once all the processes completed successfully, best would to be to also list that command in the slice of commands to execute (add that to `cmdParams`), and so the loop will also execute that as the last command. – icza Feb 18 '18 at 11:13
@RaynD I think I misunderstood, thought you have to run one last command if all other has completed. If you want to run Go code, just put it after the `for` loop, since if any previous command fails, it returns, and then the code after the loop would not be reached. – icza Feb 18 '18 at 12:15
@RaynD Please see edited answer. I simplified it as it was overly complicated. – icza Feb 18 '18 at 20:14

Running command and get the output online

1 Answers1

Linked

Related