grep several strings from gz file

Question

What is the best way to extract lines from a very large gz file that match multiple strings in a second file?

I've tried, which works for that string and surrounding:

gunzip -c /myfolder/large_file.gz | grep -B 50 "33754548"  > /myfolder/specific_linesfrom_large_files.txt

However, sometimes the strings needed are not in 50 lines near, so I attempted:

gunzip -c /myfolder/large_file.gz | grep  -F  /myfolder/multiple_strings.txt  > /myfolder/specific_linesfrom_large_files.txt

Which didn't work, any suggestions?

for example, the multiple_strings.txt file might contain:

Are you running this command in Linux or Unix? Try zgrep the same way you run grep — Krassi Em, Nov 06 '17 at 06:13
You can search for lines containing any of `StringA`, `PatternB` or `ThingC` with `grep -E "StringA|PatternB|ThingC"` by the way. — Mark Setchell, Nov 06 '17 at 10:17
I have 23 files, which are 1.4 TB total, decompressing isn't preferred. I'm running Linux. — user3403622, Nov 06 '17 at 14:28
I tried the original commands with -f (instead of -F) and zgrep. Both worked, however, -f run time was faster. — user3403622, Nov 06 '17 at 23:26

score 1 · Accepted Answer · answered Nov 06 '17 at 11:09

1

use zgrep to search into compressed files. There are also other commands like bzgrep (for bzip2 files), xzgrep etc for compressed files.

zgrep -f match_strings.txt file.gz

-f is the flag for reading the patterns from a specified file.

answered Nov 06 '17 at 11:09

thanasisp

score 0 · Answer 2 · answered Nov 06 '17 at 09:18

gunzip -c /myfolder/large_file.gz | grep -f /myfolder/multiple_strings.txt > /myfolder/specific_linesfrom_large_files.txt

Using also -x you search for exact pattern, for example if you search 123 you can match 1234, 123 etc. using -x you match only 123.

2 Answers2