remove line from file if more than one pattern appears in different line

Question

I have a file with a patter like this:

1 1 1 2 0 1 0.5
1 2 2 2 0 2 0.5
2 1 1 1 0 1 0.25
2 1 2 2 0 2 0.5
2 3 3 3 0 3 0.25

I want to remove a line if another line in the file has the same last two entries. In the example above this means that line 4 should be removed:

1 1 1 2 0 1 0.5
1 2 2 2 0 2 0.5
2 1 1 1 0 1 0.25
2 3 3 3 0 3 0.25

I can't get my head around how I can do this via the command line or with a simple awk/sed script. Any help is greatly appreciated!

score 3 · Answer 1 · answered Jun 21 '15 at 18:40

3

awk '!a[$NF,$(NF-1)]++' file

Make an array and check it hasn't already been populated with the last two fields.

answered Jun 21 '15 at 18:40

123

pasaba por aqui · Accepted Answer · 2015-06-21T18:51:46.403

1

try:

 sort -k 6 f1.txt  | uniq -f 5

assuming the original order of the lines doesn't matters.

edited Jun 21 '15 at 18:51

answered Jun 21 '15 at 18:17

pasaba por aqui

Thanks for the quick reply! I have only one file though. I want to remove the line in that file. – Aplln Jun 21 '15 at 18:24
Sorry, misunderstood. Try new solution if order of lines doesn't matters. – pasaba por aqui Jun 21 '15 at 18:44

2 Answers2