Succinct way to print all lines up until the last line that matches a given pattern

Question

I'm trying to find a succinct shell one-liner that'll give me all the lines in a file up until some pattern.

The use case is dumping all the lines in a log file until I spot some marker indicating that the server has been restarted.

Here's a stupid shell-only way that:

tail_file_to_pattern() {
    pattern=$1
    file=$2

    tail -n$((1 + $(wc -l $file | cut -d' ' -f1) - $(grep -E -n "$pattern" $file | tail -n 1 | cut -d ':' -f1))) $file
}

A slightly more reliable Perl way that takes the file on stdin:

perl -we '
    push @lines => $_ while <STDIN>;
    my $pattern = $ARGV[0];
    END {
        my $last_match = 0;
        for (my $i = @lines; $i--;) {
            $last_match = $i and last if $lines[$i] =~ /$pattern/;
        }
        print @lines[$last_match..$#lines];
    }
'

And of course you could do that more efficiently be opening the file, seeking to the end and seeking back until you found a matching line.

It's easy to print everything as of the first occurrence, e.g.:

sed -n '/PATTERN/,$p'

But I haven't come up with a way to print everything as of the last occurance.

Answer 1

Here's a sed-only solution. To print every line in $file starting with the last line that matches $pattern:

sed -e "H;/${pattern}/h" -e '$g;$!d' $file

Note that like your examples, this only works properly if the file contains the pattern. Otherwise, it outputs the entire file.

Here's a breakdown of what it does, with sed commands in brackets:

• [H] Append every line to sed's "hold space" but do not echo it to stdout [d].
• When we encounter the pattern, [h] throw away the hold space and start over with the matching line.
• When we get to the end of the file, copy the hold space to the pattern space [g] so it will echo to stdout.

Also note that it's likely to get slow with very large files, since any single-pass solution will need to keep a bunch of lines in memory.

Answer 2

Alternatively: tac "$file" | sed -n '/PATTERN/,$p' | tac

EDIT: If you don't have tac emulate it by defining

tac() {
    cat -n | sort -nr | cut -f2
}

Ugly but POSIX.

Answer 3

Load the data into an array line by line, and throw the array away when you find a pattern match. Print out whatever is left at the end.

 while (<>) {
     @x=() if /$pattern/;
     push @x, $_;
 }
 print @x;

As a one-liner:

 perl -ne '@x=() if /$pattern/;push @x,$_;END{print @x}' input-file

Answer 4

I suggest a simplification of your shell script:

tail -n +$(grep -En "$pattern" "$file" | tail -1 | cut -d: -f1) "$file"

It's substantially more concise because it:

• Uses tail's + option to print from the given line to the end, rather than having to calculate the distance from there to the end.
• Uses more concise ways of expressing command line options.

And it fixes a bug by quoting $file (so it will work on files whose names contain spaces).

Answer 5

Sed's q command will do the trick:

sed "/$pattern/q" $file

That will print all the lines until it gets to the line with the pattern. After that, sed will print that last line and quit.

Answer 6

This questions title and description don't match.

For the question's title, +1 for @David W.'s answer. Also:

sed -ne '1,/PATTERN/p'

For question in the body, you've already got some solutions.

Note that tac is probably specific to Linux. It doesn't seem to exist in BSD or OSX. If you want a solution that's multi-platform, don't rely on tac.

Of course, just about any solution is going to require that your data either be spooled in memory, or submitted once for analysis and a second time for processing. For exampel:

#!/usr/local/bin/bash

tmpfile="/tmp/`basename $0`,$$"
trap "rm $tmpfile" 0 1 2 5
cat > $tmpfile

n=`awk '/PATTERN/{n=NR}END{print NR-n+1}' $tmpfile`

tail -$n $tmpfile

Note that my use of tail is for FreeBSD. If you use Linux, you'll probably need tail -n $n $tmpfile instead.

Answer 7

Rob Davis pointed out to me what you said you wanted isn't what you really asked:

You said:

I'm trying to find a succinct shell one-liner that'll give me all the lines in a file up until some pattern.

but then at the very end of your post, you said:

But I haven't come up with a way to print everything as of the last occurance.

I've already gave you the answer for your first question. Here's a one line answer for your second question: Printing from a regular expression to the end of the file:

awk '{ if ($0 ~ /'"$pattern"'/) { flag = 1 } if (flag == 1) { print $0 } }' $file

A similar Perl one-liner:

export pattern="<regex>"
export file="<file>"
perl -ne '$flag=1 if /$ENV{pattern}/;print if $flag;' $file