4

I have lines on my standard input

$printf "C\nB\nA\n"
C
B
A

and I want to filter out lines (or substrings or regexps - whatever is easier) that appear on some other standard input:

$printf "B\nA\n"
B
A

I expect just C when entries get filtered.

I've tried with 

$printf "C\nB\nA\n" | grep -v `printf "B\nA\n"`

But then I'm getting

grep: A: No such file or directory

How can I perform filtering of standard input by lines returned by other command?

Michal Kordas
  • 10,475
  • 7
  • 58
  • 103

3 Answers3

7

You can use grep's -f option:

Matching Control
    -f FILE, --file=FILE
          Obtain patterns from FILE, one per line.
          [...]

and use the <(command) syntax for using a command's output as the content to be used:

$ printf "C\nB\nA\nA\nC\nB" | grep -vf <(printf "A\nB\n")
C
C
Roberto Bonvallet
  • 31,943
  • 5
  • 40
  • 57
2

Use awk to get control of how each line should match each other:

$ printf "C\nB\nA\n" | awk 'NF == FNR { a[$0] = 1; next } a[$0]' \
  <(printf "A\nB\n") -

By changing a[$0] you can define how each should match, i.e to print lines from file1.txt which first column are in file2.txt:

$ awk 'NF == FNR { a[$0] = 1; next } a[$1]' file2.txt file1.txt
#                                      ^ Print if column 1 from file1.txt
#                                        is in file2.txt

To print lines from file1.txt which are contained in column one from file2.txt:

$ awk 'NF == FNR { a[$1] = 1; next } a[$0]' file2.txt file1.txt
#                                      ^ Print if line from file1.txt match
#                    ^ Store column one from file2.txt
Andreas Louv
  • 46,145
  • 13
  • 104
  • 123
0

U can use

printf "C\nB\nA\n" | grep -v C

and subsequently

printf "C\nB\nA\n" | grep -v C | grep -v B

then

printf "C\nB\nA\n" | grep -v A | grep -v B

if you want to do it from a file assuming file contains

C
B
A

then:

cat file | grep -v B | grep -v A

will print

C

if you want to filter by only piping once

cat file | grep -v 'B\|A'
C
maco1717
  • 823
  • 10
  • 26
  • 2
    The thing is that my both inputs are dynamic (based on `curl` result) and I don't want to create any intermediate files. – Michal Kordas Nov 10 '16 at 15:54
  • that should work too, by inputs you mean the fields to filter? or what you are piping? if it's what you are piping then just change 'cat file' to whatever you are want to run if ifs the string to filter where would this "variable" come from? – maco1717 Nov 10 '16 at 16:07
  • Lines to filter and the filter itself is dynamic and separated by newlines (not `\|`). So `\`curl input1\` | grep -v \`curl input2\`` doesn't work – Michal Kordas Nov 10 '16 at 16:25
  • from the grep man: In basic regular expressions the metacharacters ?, +, {, |, (, and ) lose their special meaning; instead use the backslashed versions \?, \+, \{, \|, \(, and \). - is not the separation by another "variable" or string to match... – maco1717 Nov 10 '16 at 16:38
  • that should work. I've tried this... "read -p "where from: " -r url ; read -p "delete: " -r string; curl $url | grep -v $string" whith url google.com and string BODY and it deletes the whole line where it finds the match... – maco1717 Nov 10 '16 at 17:04
  • In my case I need to filter out multiple patterns based on dynamic input, while above you provided example how to filter out just a single pattern. – Michal Kordas Nov 10 '16 at 17:46
  • Try this... `read -p "where from: " -r url ; read -p "delete: " -r string; read -p "delete2: " -r string2; curl $url | grep -v "$string\|$string2"` – maco1717 Nov 10 '16 at 19:42