14

I'm running a command line like this:

filename_listing_command | xargs -0 action_command

Where filename_listing_command uses null bytes to separate the files -- this is what xargs -0 wants to consume.

Problem is that I want to filter out some of the files. Something like this:

filename_listing_command | sed -e '/\.py/!d' | xargs ac

but I need to use xargs -0.

How do I change the line separator that sed wants from newline to NUL?

bstpierre
  • 30,042
  • 15
  • 70
  • 103

4 Answers4

22

If you've hit this SO looking for an answer and are using GNU sed 4.2.2 or later, it now has a -z option which does what the OP is asking for.

Idcmp
  • 659
  • 1
  • 8
  • 19
7

Pipe it through grep:

filename_listing_command | grep -vzZ '\.py$' | filename_listing_command

The -z accepts null terminators on input and the -Z produces null terminators on output and the -v inverts the match (excludes).

Edit:

Try this if you prefer to use sed:

filename_listing_command | sed 's/[^\x0]*\.py\x0//g' | filename_listing_command
Dennis Williamson
  • 346,391
  • 90
  • 374
  • 439
  • Your `sed` solution still breaks on `\n`, so if a filename contains `\n` and ends in `.py`, only the part after `\n` will be deleted. For example, `printf '%s\0' $'abc\ndef.py' | sed 's/[^\x0]*\.py\x0//g'` will print `abc\n`. This defeats the purpose of using null terminators. – musiphil Jan 24 '14 at 03:13
  • In my case, my command (for `\n` terminators) was `sed -ne 's/^UU //p'`, and I couldn't convert it to a `grep` command easily. – musiphil Jan 24 '14 at 03:18
  • @musiphil: Try: `grep -PzZo '^(?<=UU ).*'` – Dennis Williamson Jan 24 '14 at 14:22
  • @DennisWilliamson: Thanks, but it doesn't work. You can try this test case: `printf '%s\0' 'UU abc.py' | cmd` with your `grep` as `cmd`, and `abc.py\0` is expected but `grep` prints nothing (and leaves an exit status of 1). – musiphil Jan 24 '14 at 20:57
  • 1
    @musiphil: Sorry, put the carat inside the lookbehind: `grep -PzZo '(?<=^UU ).*'`. Unfortunately, the `-o` causes a newline to be added. Without it, the output is null-terminated, but the "UU " isn't stripped. I think that a feature should be added so that `-z` combined with `-o` outputs nulls. – Dennis Williamson Jan 25 '14 at 14:58
  • @DennisWilliamson: Thanks. It seems that `-o` combined with `-z` prevents `grep` from treating newlines as ordinary characters. I tried `printf '%s\0' $'UU abc\ndef.py' | grep -PzZo '(?<=^UU ).*'` and only `abc\n` is printed; `printf '%s\0' $'UU abc\ndef.py' | grep -PzZ '(?<=^UU ).*'` correctly prints the whole name, but as you mentioned the initial `UU ` isn't stripped. I believe this is a bug, but they say that `-P` is _highly experimental_ anyway. :-( – musiphil Jan 27 '14 at 02:46
  • Pitty that `grep -zZ` options are not in POSIX, which holds also for `\x0` in `sed`. These are Gnu extensions. E.g. on Mac, you are doomed. – Palec Jan 27 '14 at 02:52
  • Perhaps you are looking for `git status -sz | grep -z '^UU ' | sed -rz 's/^UU (.*)/\1/' | xargs -0 $EDITOR`. Works with `sed (GNU sed) 4.2.2` – Tom Hale Oct 18 '16 at 15:01
1

If none of your file names contain newline, then it may be easier to read a solution using GNU Parallel:

filename_listing_command | grep -v '\.py$' | parallel ac

Learn more about GNU Parallel http://www.youtube.com/watch?v=OpaiGYxkSuQ

Ole Tange
  • 1,990
  • 16
  • 10
0

With help of Tom Hale and that answer we have: sed -nzE "s/^$PREFIX(.*)/\1/p"

Netsu
  • 355
  • 4
  • 13