68

I have been trying to get the head utility to display all but the last line of standard input. The actual code that I needed is something along the lines of cat myfile.txt | head -n $(($(wc -l)-1)). But that didn't work. I'm doing this on Darwin/OS X which doesn't have the nice semantics of head -n -1 that would have gotten me similar output.

None of these variations work either.

cat myfile.txt | head -n $(wc -l | sed -E -e 's/\s//g')
echo "hello" | head -n $(wc -l | sed -E -e 's/\s//g')

I tested out more variations and in particular found this to work:

cat <<EOF | echo $(($(wc -l)-1))
>Hola
>Raul
>Como Esta
>Bueno?
>EOF
3

Here's something simpler that also works.

echo "hello world" | echo $(($(wc -w)+10))

This one understandably gives me an illegal line count error. But it at least tells me that the head program is not consuming the standard input before passing stuff on to the subshell/command substitution, a remote possibility, but one that I wanted to rule out anyway.

echo "hello" | head -n $(cat && echo 1)

What explains the behavior of head and wc and their interaction through subshells here? Thanks for your help.

Gilles 'SO- stop being evil'
  • 104,111
  • 38
  • 209
  • 254
gkb0986
  • 3,099
  • 1
  • 24
  • 22

7 Answers7

99

head -n -1 will give you all except the last line of its input.

devnull
  • 118,548
  • 33
  • 236
  • 227
dunc123
  • 2,595
  • 1
  • 11
  • 11
85

head is the wrong tool. If you want to see all but the last line, use:

sed \$d

The reason that 

# Sample of incorrect code:
echo "hello" | head -n $(wc -l | sed -E -e 's/\s//g')

fails is that wc consumes all of the input and there is nothing left for head to see. wc inherits its stdin from the subshell in which it is running, which is reading from the output of the echo. Once it consumes the input, it returns and then head tries to read the data...but it is all gone. If you want to read the input twice, the data will have to be saved somewhere.

mahemoff
  • 44,526
  • 36
  • 160
  • 222
William Pursell
  • 204,365
  • 48
  • 270
  • 300
  • Thanks @William. I could use `sed`, but in this question, I'm specifically trying to understand why the approach using `head` doesn't work. – gkb0986 Aug 08 '13 at 13:55
  • +1 Why on Earth was this downvoted? The `sed` script is absolutely the correct, portable solution, and I find no flaw in the explanation. – tripleee Aug 08 '13 at 13:56
  • The downvote was reasonable. It occurred when the answer was nothing but the `sed` solution which did not address the question at all. – William Pursell Aug 08 '13 at 13:57
  • 3
    @aerijman `\$d` is another way of writing `'$d'`. The `$` addresses the last line, and the command `d` is applied to that line. That is, it deletes the last line. – William Pursell Jan 22 '20 at 03:36
  • On Windows use `sed $d` (no need to escape `$`). – dzieciou Apr 09 '20 at 11:39
  • 1
    @aerijman `sed` is not very good at addressing lines from the end of the file. With gnu head (from coreutils 8.30) you can do `head -n -$n` (eg, `head -n -5` to see all but the last 5 lines). – William Pursell May 14 '20 at 16:20
14

Using sed:

sed '$d' filename

will delete the last line of the file.

$ seq 1 10 | sed '$d' 
1
2
3
4
5
6
7
8
9
devnull
  • 118,548
  • 33
  • 236
  • 227
10

For Mac OS X specifically, I found an answer from a comment to this Q&A.

Assuming you are using Homebrew, run brew install coreutils then use the ghead command:

cat myfile.txt | ghead -n -1

Or, equivalently:

ghead -n -1 myfile.txt

Lastly, see brew info coreutils if you'd like to use the commands without the g prefix (e.g., head instead of ghead).

Community
  • 1
  • 1
Jimothy
  • 9,150
  • 5
  • 30
  • 33
6
cat myfile.txt | echo $(($(wc -l)-1))

This works. It's overly complicated: you could just write echo $(($(wc -l)-1)) <myfile.txt or echo $(($(wc -l <myfile.txt)-1)). The problem is the way you're using it.

cat myfile.txt | head -n $(wc -l | sed -E -e 's/\s//g')

wc consumes all the input as it's counting the lines. So there is no data left to read in the pipe by the time head is started.

If your input comes from a file, you can redirect both wc and head from that file.

head -n $(($(wc -l <myfile.txt) - 1)) <myfile.txt

If your data may come from a pipe, you need to duplicate it. The usual tool to duplicate a stream is tee, but that isn't enough here, because the two outputs from tee are produced at the same rate, whereas here wc needs to fully consume its output before head can start. So instead, you'll need to use a single tool that can detect the last line, which is a more efficient approach anyway.

Conveniently, sed offers a way of matching the last line. Either printing all lines but the last, or suppressing the last output line, will work:

sed -n '$! p'
sed '$ d'
fedorqui
  • 275,237
  • 103
  • 548
  • 598
Gilles 'SO- stop being evil'
  • 104,111
  • 38
  • 209
  • 254
2

Here is a one-liner that can get you the desired output, and it can be used more generally for getting all lines from a file except the last n lines.

grep -n "" myfile.txt \ # output the line number for each line
| sort -nr \            # reverse the file by using those line numbers
| sed '1,4d' \          # delete first 4 lines (last 4 of the original file)
| sort -n \             # reverse the reversed file (correct the line order)
| sed 's/^[0-9]*://'    # remove the added line numbers

Here is the above command in an actual single line and runnable (can't execute the above due to the added comments):

grep -n "" myfile.txt | sort -nr | sed '1,4d' | sort -n | sed 's/^[0-9]*://'

It's a little cumbersome, and this problem can be solved with more comprehensive commands like ghead, but when you can't or don't want to download such tools, it's nice to be able to do this with the more basic options. I've been in situations where it's simply not an option to get better tools.

daneorie
  • 21
  • 2
0
awk 'NR>1{print p}{p=$0}'

For this job, an awk one-liner is a bit longer than a sed one.

Esoteric Screen Name
  • 6,082
  • 4
  • 29
  • 38
Kent
  • 189,393
  • 32
  • 233
  • 301