How to add a NUL character separator using AWK in bash?

Question

I have a function which outputs some file paths, I need these paths are separated by NUL charachter instead of new line \n character. I tried following code:

function myfunc
{
  declare -a DUPS
  # some commands to fill DUPS with appropriate file/folder paths
  ( for i in "${DUPS[@]}"; do echo "$i"; done )|sort|uniq|awk 'BEGIN{ORS="\x00";} {print substr($0, index($0, $2))}'
}

But if I pipe its output to hexdump or hd, no NUL character is diplayed. It seems that NUL character is not included in the awk output:

myfunc | hd

Will print:

00000000  2f 70 61 74 68 2f 6e 75  6d 62 65 72 2f 6f 6e 65  |/path/number/one|
00000010  2f 2f 70 61 74 68 2f 6e  75 6d 62 65 72 2f 74 77  |//path/number/tw|
00000020  6f 2f 2f 70 61 74 68 2f  6e 75 6d 62 65 72 2f 74  |o//path/number/t|
00000030  68 72 65 65 2f                                    |hree/|
00000035

My awk version is:

~$ awk -W version
mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

compiled limits:
max NF             32767
sprintf buffer      2040

Also any solution with other commands such as sed is acceptable for me.
NOTE: My question is not duplicate of enter link description here, because it asks for a solution that works on different machines with different awks. But I just need a solution that works on my own machine, so I could use any version of awk that could be installed on Ubuntu 14.04.

to install gawk, `sudo apt-get install gawk`. Or you can use the `printf` solution in my answer to get `mawk` to do what you want. — rici, Aug 30 '15 at 21:56

score 2 · Answer 1 · answered Aug 30 '15 at 04:41

Gnu Awk v4.0.1 works just fine with your original program, but all the other awks I have kicking around (mawk, original-awk and busybox awk) produce the same NUL-free output as you seem to be experiencing. It appears that with those awks, using either print or printf to print a string with embedded NULs causes the NUL to be treated as a string terminator.

However, mawk and original-awk will output a real NUL if you use printf "%s",0;. So if you are using one of those, you could set ORS to the empty string and add {printf "%s", 0;} to the end of your awk program. (You'd need other more invasive modifications if your awk program uses next).

I don't know any way to convince busybox awk to print a NUL byte, so if that is what you are using you might want to consider choosing a real awk.

I installed `Gnu Awk v4.0.1` and replaced `awk` with `gawk` in my code in the question. But the result is the same! — PHP Learner, Aug 31 '15 at 11:38
@PHPLearner: You got me. On my system, with gawk 4.0.1, it works perfectly. By the way, you can simplify considerably: `printf '%s\n' "${DUPS[@]}" | sort -u | awk ...` — rici, Aug 31 '15 at 15:09

How to add a NUL character separator using AWK in bash?

1 Answers1