How to change a file on the fly along a pipe?
I'm probably looking for a way to buffer a file at the beginning of a pipe, which in contrast to this:
cat foo.txt | grep bar > foo.txt
... would preserve the input data from destruction by the pipe itself. Is there such a buffer in stock?
Try using sponge from moreutils like this:
sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd
It collects the whole input before writing to it's output.
I would guess sed still might create the temp file, but the following might do what you want? (Using strace on this might show you if sed creates a temp file or not).
sed -i '/bar/!d' foo.txt
The exclamation inverts the match, d is for delete, so this removes all lines that don't have bar in them.
Use >> to preserve the contents.
cat foo.txt | grep bar >> foo.txt
Now that will append to the file.
AFAIK, there is no direct way to prepend data to a file in shell. If you want to prepend you may need to use a temporary file in between.
Depending on how complex your command line is, you may get mileage out of
cat foo.txt | grep bar | tee -a foo.txt
The best way to do this is to remember how files work in unix -- the file exists as long as there is a link to the file (directory or process opening the file).
So, open the file, delete that directory entry, then run your process that writes to a new directory entry with the same name but linked to a different inode.
Lastly, the nice thing about this is that it works for any sort of pipeline operation, not just things that can do the temporary file buffer hack.
{ rm file ; sed -e s/this/that/g > file ; } < file
A word of warning -- this may cause disaster if you must have your rewrite be an atomic operation. unix doesn't offer clean ways of locking files, especially not at shell level. This was an issue in the dark ages if you were doing something like editing a password file on a busy system that was running something like NIS. Any time more than one process is reading / writing to a file, be very careful if your system is busy or important.
The only operations that are 100% certain to be atomic are directory entry manipulations -- rm / ln / mv (on the same filesystem).
So now things get longer and uglier..
The math part of this requires a posix shell or similarly extended bourne shell
c=0
while
f=$SECONDS
test -f file.$c.tmp
do
c=$(($c+1))
done
grep stuff < file > file.$c.tmp && mv file.$c.tmp file
Ooops, UUOC, useless use of cat :) you don't need cat there :
grep something somefile > someotherfile
As sekenre said, there is sponge. Or maybe you may want to try perl inline edition ( -i switch ) :
print only matching output:
perl -ni -e 'm/foo/g and print ' bar.txt
replace "foo" with "bar" in the file :
perl -pi -e 's/foo/bar/g' bar.txt
