0

I have very big text files(~50,000) over which i have to do some text processing. Basically run multiple grep commands. When i run it manually it returns in an instant , but when i do the same in a bash script - it takes a lot of time. What am i doing wrong in below bash script. I pass the names of files as command line arguments to script

Example Input data : 

BUSINESS^GFR^GNevil
PERSONAL^GUK^GSheila

Output that should come in a file - BUSINESS^GFR^GNevil

It starts printing out the whole file on the terminal after quite some while. How do i suppress the same?

#!/bin/bash
cat $2 | grep BUSINESS
nikel
  • 3,402
  • 11
  • 45
  • 71
  • Why do you need to `cat` at all? Why not just `grep` directly? – Daniel Dec 26 '14 at 14:03
  • 2
    What is `$($(cat $2) ` also, to grep a file don't use `cat f | grep 'something'` but `grep 'something' f` – fredtantini Dec 26 '14 at 14:03
  • I don't think you need `cat` here - just specify the file in the call to grep. – Dason Dec 26 '14 at 14:04
  • Okay. but when I can use the same at command line why not in a script - cat test | grep BUSINESS | grep '^GFR|^GDE' – nikel Dec 26 '14 at 14:05
  • Apart from [`UUOC`](http://www.smallo.ruhr.de/award.html#cat), this is wrong too. `cat $2 | grep ... ` might have at least given you correct results. `$(cat $2) | grep BUSINESS` is plain wrong. Essentially, the contents of `$2` would be run as command... – anishsane Dec 26 '14 at 14:17
  • i pass the filenames as cmd line argument number 2, so thats what i wanted – nikel Dec 26 '14 at 14:21
  • ^^ No, `cat file | command` is different from `$(cat file) | command`. e.g. Create a file containing just a single line : `ls` & check output of `cat file | grep .` vs `$(cat file) | grep .`... – anishsane Dec 26 '14 at 15:20

1 Answers1

1

Do NOT use cat with program that can read file itself.
It slows thing down and you lose functionality:

grep BUSINESS test | grep '^GFR|^GDE'

Or you can do like this with awk

awk '/BUSINESS/ && /^GFR|^GDE/' test
Jotne
  • 40,548
  • 12
  • 51
  • 55
  • also, how do i suppress the grep output, I changed it as below : $(grep BUSINESS $2| grep -E "$countries")>>"$Output_File" – nikel Dec 26 '14 at 14:11
  • Do you like the output to a file? Give example data in your original post and how you like the output to be. – Jotne Dec 26 '14 at 14:13