0
File1:
foo
bar
baz

File2:
bar.patch

grep -f file1 file2 > file1

Expected result, file1 contains bar.patch, instead it is empty.

How is grep processing the input file such that I cannot redirect to it?

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
Buoy
  • 307
  • 1
  • 9

1 Answers1

2

Redirection > happens before the command is executed, hence the grep sees empty file and returns nothing. If you really want to write to the same file you can use the sponge command (it is part of the moreutils). It buffers the input and writes to the output file only after the end of data. Example:

grep -f file1 file2 | sponge file1

Check the bash manual for details about redirection:

Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell. Redirection allows commands’ file handles to be duplicated, opened, closed, made to refer to different files, and can change the files the command reads from and writes to. Redirection may also be used to modify file handles in the current shell execution environment. The following redirection operators may precede or appear anywhere within a simple command or may follow a command. Redirections are processed in the order they appear, from left to right.

Or just redirect to a temporary file and then mv it over the file1.

grep -f file1 file2 > file3
mv -f file3 file1
Dzienny
  • 3,295
  • 1
  • 20
  • 30
  • 2
    There's no guarantee that `tee` won't open `file1` for writing (thus truncating it) before `grep` has finished reading git. It's just as bad as the output redirection. `sponge` or the temporary file is necessary. – chepner Feb 09 '22 at 20:57
  • Oh, right, `sponge` is only buffering the output of `grep`, not affecting what `tee` does to `file1`. – chepner Feb 09 '22 at 21:03
  • 2
    _nod_. `grep -f file1 file2 | sponge file1` is fine. `grep -f file1 file2 | sponge | tee file1` is not. – Charles Duffy Feb 09 '22 at 21:04
  • Indeed, I overlooked the parallel execution of the pipe. Good points. – Dzienny Feb 09 '22 at 21:19