-1

Is it possible to concatenate the headers lines in a file with the output from a filter using grep? Perhaps using the cat command or something else from GNU's coreutils?

In particular, I have a tab delimited file that roughly looks like the following:

var1   var2   var3
1      MT     500
30     CA     40000
10     NV     1240   
40     TX     500   
30     UT     35000
10     AZ     1405
35     CO     500
15     UT     9000
1      NV     1505
30     CA     40000
10     NV     1240

I would like to select from lines 2 - N all lines that contain "CA" using grep and also to place the first row, the variable names, in the first line of the output file using GNU/Linux commands.

The desired output for the example would be:

var1   var2   var3
30     CA     40000
35     CA     65000
15     CA     2500

I can select the two sets of desired output with the following lines of code.

head -1 filename
grep -E CA filename

My initial idea is to combine the output of these commands using cat, but I have not been successful so far.

lmo
  • 37,904
  • 9
  • 56
  • 69
  • 2
    I don't understand the question, could you provide an example of input and output? – martin May 13 '15 at 12:46
  • @martin. Sorry for the delay. Lithis answered my question. I will elaborate the question, however, as I haven't seen it asked in this forum before (though I did not know about && in bash). – lmo May 13 '15 at 23:50

3 Answers3

2

If you're running the commands from a shell (including shell scripts), you can run each command separately and redirect the output:

head -1 filename > outputfile
grep -E CA filename >> outputfile

The first line will overwrite outputfile, because a single > was used. The second line will append to outputfile, because >> was used.

If you want to do this in a single command, the following worked in bash:

(head -1 filename && grep -E CA filename) > outputfile

If you want the output to go to standard output, leave off the parenthesis and redirection:

head -1 filename && grep -E CA filename
Lithis
  • 1,327
  • 8
  • 14
  • I have never seen the && syntax. I will have too look into it. Thank you for the info. This will be quite useful. – lmo May 13 '15 at 23:46
1

It's not clear what you're looking for, but perhaps just:

{ head -1 filename; grep -E CA filename; } > output

or

awk 'NR==1 || /CA/' filename > output

But another interpretation of your question is best addressed using sed or awk. For example, to print lines 5-9 and line 14, you can do:

sed -n -e 5,9p -e 14p

or

awk '(NR >=5 && NR <=9) || NR==14'
William Pursell
  • 204,365
  • 48
  • 270
  • 300
  • I have not yet ventured into the world of sed/awk. Are these tools preferable relative to grep for performing a simple data filter ( ie, row selection)? My assumption was that as these tools are more flexible, they also have more overhead --> using the GNU core utilities, is the most efficient though less can be accomplished. – lmo May 15 '15 at 12:43
0

I just came across a method that uses the cat command.

cat <(head -1 filename) <(grep -E CA filename) > outputfile

This site, tldp.org, calls the <(command) syntax "process substitution."

It is unclear to me what method would be more efficient in terms of memory / speed, but this is testable.

lmo
  • 37,904
  • 9
  • 56
  • 69