Stack Overflow
Stack Overflow

Questions tagged [pipe]

A pipe is an interprocess connection between file descriptors of two processes. A pipe is created with the POSIX pipe() function (from ). Shells create pipes between processes if the "|" symbol is used: "cmd1 | cmd2" directs the output of cmd1 to the input of cmd2. On Windows use CreatePipe(). This mechanism redirects standard input, standard output, and standard error into the calling process in .NET and Java.

Every program that runs on the command line has three data streams connected to it: STDIN(0) - standard input, STDOUT (1) - standard output - data printed by the program, STDERR(2) - standard error.

There are ways to connect streams between programs and files called piping and redirections.

Piping is a mechanism for sending data from one program to another using the "|" operator. The operator feeds the output from the program on the left as input to the program on the right.

Example 

$ cat two_columns
column1:cloth
column2:strawberries
column3:fish
column4:chocolate
column5:punch cards
$ cat two_columns | awk -F: '{print $1}'
column1
column2
column3
column4
column5 
$ cat two_columns | awk -F: '{print "HAS: " $2}'
HAS: cloth
HAS: strawberries
HAS: fish
HAS: chocolate
HAS: punch cards

Useful Links

9809 questions
96
votes
5 answers

What are the differences between R's new native pipe `|>` and the magrittr pipe `%>%`?

In R 4.1 a native pipe operator was introduced that is "more streamlined" than previous implementations. I already noticed one difference between the native |> and the magrittr pipe %>%, namely 2 %>% sqrt works but 2 |> sqrt doesn't and has to be…
sieste
  • 8,296
  • 3
  • 33
  • 48
96
votes
4 answers

Use pipe operator %>% with replacement functions like colnames()<-

How can I use the pipe operator to pipe into replacement function like colnames()<- ? Here's what I'm trying to do: library(dplyr) averages_df <- group_by(mtcars, cyl) %>% summarise(mean(disp), mean(hp)) colnames(averages_df) <- c("cyl",…
Alex Coppock
  • 2,122
  • 3
  • 15
  • 31
90
votes
5 answers

Linux why can't I pipe find result to rm?

sorry if this is a noobie question but I can't find a good answer. To find then remove something I can use find . -name ".txt" -exec rm "{}" \; But why can't I just pipe the results to rm like find . -name ".txt" | rm like I would pipe it to…
user1297061
  • 1,531
  • 2
  • 13
  • 15
87
votes
6 answers

How can I send the stdout of one process to multiple processes using (preferably unnamed) pipes in Unix (or Windows)?

I'd like to redirect the stdout of process proc1 to two processes proc2 and proc3: proc2 -> stdout / proc1 \ proc3 -> stdout I tried proc1 | (proc2 & proc3) but it doesn't seem to work, i.e. echo 123 | (tr 1 a &…
secr
  • 2,743
  • 3
  • 20
  • 20
84
votes
8 answers

pipe stdout and stderr to two different processes in shell script?

I've a pipline doing just command1 | command2 So, stdout of command1 goes to command2 , while stderr of command1 go to the terminal (or wherever stdout of the shell is). How can I pipe stderr of command1 to a third process (command3) while…
user964970
81
votes
12 answers

Delete a list of files with find and grep

I want to delete all files which have names containing a specific word, e.g. "car". So far, I came up with this: find|grep car How do I pass the output to rm?
Magnus
  • 1,550
  • 4
  • 14
  • 33
80
votes
6 answers

File not found error when launching a subprocess containing piped commands

I need to run the command date | grep -o -w '"+tz+"'' | wc -w using Python on my localhost. I am using subprocess module for the same and using the check_output method as I need to capture the output for the same. However it is throwing me an error…
h4ck3d
  • 6,134
  • 15
  • 51
  • 74
80
votes
4 answers

Why piping input to "read" only works when fed into "while read ..." construct?

I've been trying to read input into environment variables from program output like this: echo first second | read A B ; echo $A-$B And the result is: - Both A and B are always empty. I read about bash executing piped commands in sub-shell and…
huoneusto
  • 1,164
  • 1
  • 8
  • 19
78
votes
3 answers

How can I redirect all output to /dev/null?

I want to run a program (google-chrome) in the background, but prevent it from outputting any messages to the terminal. I tried doing this: google-chrome 2>&1 1>/dev/null & However, the terminal still fills up without messages…
Benubird
  • 18,551
  • 27
  • 90
  • 141
78
votes
3 answers

How to pipe multiple commands into a single command in the shell? (sh, bash, ...)

How can I pipe the stdout of multiple commands to a single command? Example 1: combine and sort the output of all three echo commands: echo zzz; echo aaa; echo kkk desired output: aaa kkk zzz Example 2: rewrite the following so that all the…
Rob Bednark
  • 25,981
  • 23
  • 80
  • 125
77
votes
6 answers

When should I use GCC's -pipe option?

The GCC 4.1.2 documentation has this to say about the -pipe option: -pipe Use pipes rather than temporary files for communication between the various stages of compilation. This fails to work on some systems where the assembler is unable to read…
Rob Kennedy
  • 161,384
  • 21
  • 275
  • 467
75
votes
9 answers

How do I use subprocess.Popen to connect multiple processes by pipes?

How do I execute the following shell command using the Python subprocess module? echo "input data" | awk -f script.awk | sort > outfile.txt The input data will come from a string, so I don't actually need echo. I've got this far, can anyone…
Tom
  • 42,844
  • 35
  • 95
  • 101
74
votes
2 answers

Piping a file into docker run

I need to pipe (inject) a file or some data into docker as part of the run command and have it written to a file within the container as part of the startup. Is there best practise way to do this ? I've tried this. cat data.txt | docker run -a…
user1513388
  • 7,165
  • 14
  • 69
  • 111
74
votes
5 answers

Concatenate in bash the output of two commands without newline character

What I need: Suppose I have two commands, A and B, each of which returns a single-line string (i.e., a string with no newline character, except possibly 1 at the very end). I need a command (or sequence of piped commands) C that concatenates the…
synaptik
  • 8,971
  • 16
  • 71
  • 98
72
votes
4 answers

How to pipe input to python line by line from linux program?

I want to pipe the output of ps -ef to python line by line. The script I am using is this (first.py) - #! /usr/bin/python import sys for line in sys.argv: print line Unfortunately, the "line" is split into words separated by whitespace. So,…
CodeBlue
  • 14,631
  • 33
  • 94
  • 132
1 2 3
…
99 100