1

i have a command that potentially outputs a lot of data to stdout and I need to upload that via ftp to a remote location.

I found this question Upload output of a program directly to a remote file by ftp and I really liked the idea of redirecting the output into a named pipe and then read junks from it. however as soon as I read the first chunk via dd the command inputing into the pipe just exits and there is no more data to read from the pipe.

to test this i created a fifo

#> mkfifo fifo

then I wrote into the fifo on one shell:

#> echo bla  > fifo

and on another shell i read from it:

#> dd if=fifo of=spool.1 bs=1 count=1

it outputs the first byte into spool.1 and then the command writing into the pipe exits and I can't read the remaining data from the pipe.

I would like to read the next chunk from that pipe but I can't figure out what I am doing wrong

any idea how to keep that pipe open until all data is read from it?

user3347114
  • 175
  • 7

1 Answers1

3

dd needs to read from standard input, rather than opening and closing the pipe itself, to keep the write end open for echo. Once the write end is closed, you can't open the read end again.

For example

{ 
  dd of=spool.1 bs=1 count=1
  dd of=spool.2 bs=2 count=2
  dd of=spool.2 bs=2 count=2
} < fifo

fifo is opened once for the compound command {...}, and each call to dd inherits the same open file descriptor without closing it.

chepner
  • 497,756
  • 71
  • 530
  • 681
  • thx - that solves that issue (just had to replace the curly brackets with round ones for some reason). however this even goes on if there is no data in the pipe anymore - is there a way of looping until the command puting data into the pipe has finished? – user3347114 Dec 17 '18 at 15:31
  • Note sure why you would need parentheses; did you try to put this on one line? `{...}` requires a `;` after *every* command, including the last one. As for the second question, if the pipe is empty, the reader will block until the writer adds more data (or the writer closes its end); and if the pipe is full, the writer will block until a reader consumes some data (or closes its end of the pipe). That's just how pipes work. – chepner Dec 17 '18 at 15:34