Bash - remove all lines beginning with 'P'

Question

I have a text file that's about 300KB in size. I want to remove all lines from this file that begin with the letter "P". This is what I've been using:

> cat file.txt | egrep -v P*

That isn't outputting to console. I can use cat on the file without another other commands and it prints out fine. My final intention being to:

> cat file.txt | egrep -v P* > new.txt

No error appears, it just doesn't print anything out and if I run the 2nd command, new.txt is empty. I should say I'm running Windows 7 with Cygwin installed.

qnd: Regex is wrong. `P*` says "match any number (even 0) of `P`" so it matches every single line. — doubleDown, Jun 20 '13 at 09:08

Édouard Lopez · Accepted Answer · 2013-06-20T09:22:21.510

16

Explanation

use ^ to anchor your pattern to the beginning of the line ;
delete lines matching the pattern using sed and the d flag.

Solution #1

cat file.txt | sed '/^P/d'

Better solution

Use sed-only:

sed '/^P/d' file.txt > new.txt

edited Jun 20 '13 at 09:22

answered Jun 20 '13 at 08:36

Édouard Lopez

40,270
28
126
178

1

@MeJ yep but I prefer to remove flag that can be of destructive nature as people sometimes copy/paste command – Édouard Lopez Mar 10 '15 at 15:39
Thank you for adding the explanation. – yaserso Nov 11 '15 at 13:28

Zulu · Answer 2 · 2014-07-17T09:39:55.157

10

With awk:

awk '!/^P/' file.txt

Explanation

The condition starts with an ! (negation), that negates the following pattern ;
- /^P/ means "match all lines starting with a capital P",
So, the pattern is negated to "ignore lines starting with a capital P".
Finally, it leverage awk's behavior when { … } (action block) is missing, that is to print the record validating the condition.

So, to rephrase, it ignores lines starting with a capital P and print everything else.

Note

sed is line oriented and awk column oriented. For your case you should use the first one, see Edouard Lopez's reponse.

edited Jul 17 '14 at 09:39

answered Jun 20 '13 at 10:09

Zulu

8,765
9
49
56

As a toy use case for `awk`, say we want to print `: ` in all uncommented lines (i.e., not starting with `#`). Here we're actually operating on fields instead of just printing the full lines, so `awk` makes sense: `awk '!/^#/ {print $1": "$4}' file.txt` – tdy Jun 18 '21 at 04:36

Grzegorz Żur · Answer 3 · 2013-06-20T09:42:50.563

2

Use start of line mark and quotes:

 cat file.txt | egrep -v '^P.*'

P* means P zero or more times so together with -v gives you no lines

^P.* means start of line, then P, and any char zero or more times

Quoting is needed to prevent shell expansion.

This can be shortened to

egrep -v ^P file.txt

because .* is not needed, therefore quoting is not needed and egrep can read data from file.

As we don't use extended regular expressions grep will also work fine

grep -v ^P file.txt

Finally

grep -v ^P file.txt > new.txt

edited Jun 20 '13 at 09:42

answered Jun 20 '13 at 08:36

Grzegorz Żur

47,257
14
109
105

5

Note that it is not necessary to use `cat file | egrep ...`, this is better: `egrep .... file`. – fedorqui Jun 20 '13 at 09:17
2

The `.*` is superfluous as well. Drop that, and you don't even need to quote. Just use `grep -v ^P "${old_file}" > "${new_file}"`. – Adrian Frühwirth Jun 20 '13 at 09:22
This will make a complete answer now. – Grzegorz Żur Jun 20 '13 at 09:46

score 2 · Answer 4 · answered Jun 20 '13 at 09:05

2

Use sed with inplace substitution (for GNU sed, will also for your cygwin)

sed -i '/^P/d' file.txt

BSD (Mac) sed

sed -i '' '/^P/d' file.txt

answered Jun 20 '13 at 09:05

bartimar

3,374
3
30
51

2

I do not recommend inplace replacement as it might lead to data-loss. Especially on wide-public site such as SO. – Édouard Lopez Jun 20 '13 at 09:18
2

@EdouardLopez Just use `sed -i .bak -e '/^P/d' file.txt` to create a backup copy – l0b0 Jun 20 '13 at 09:23
@l0b0 yep, but it should be avoided when noobs are the target audience – Édouard Lopez Jun 20 '13 at 09:25
Noobs will make `sed '/^P/d' file.txt > file.txt` which is even worse :)) – bartimar Jun 20 '13 at 14:04

score 1 · Answer 5 · answered Jun 20 '13 at 08:42

1

This works:

cat file.txt | egrep -v -e '^P'

-e indicates expression.

answered Jun 20 '13 at 08:42

Prasanth

5,230
2
29
61

Unecessary use of cat. `egrep -v -e '^P' file.txt` – mak Mar 31 '21 at 11:04

Bash - remove all lines beginning with 'P'

5 Answers5

Explanation

Solution #1

Better solution

Explanation

Note

Linked

Related