Reading file line by line using awk and for loop

Question

File contains compiler generated warnings

warnings.txt(with 8 lines)

 Line1  "/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp", line 1657: error #4296-D: arithmetic operation on boolean type
 Line2{ return soap->count - soap->buflen + soap->bufidx - (soap->ahead != 0);
 Line3
 Line4  
 Line5  "/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp", line 4136: warning #3348-D: declaration hides variable "err" (declared at line 3
 Line6 700)
 Line7          { int err = SSL_get_error(soap->ssl, r);
 Line8                ^

I want to read this file line by line using awk as while loop takes much time to process

Two ways i tried:

for line in `awk '{print $0}' warnings.txt`
do
  echo $line
done

and,

for line in `awk '{BEGIN{ORS="\n";}{print $0}' warnings.txt`
do
  echo $line
done

Expected Output:

Same as above(i.e. content of file line by line)

 Line1  "/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp", line 1657: error #4296-D: arithmetic operation on boolean type
 Line2{ return soap->count - soap->buflen + soap->bufidx - (soap->ahead != 0);
 Line3
 Line4  
 Line5  "/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp", line 4136: warning #3348-D: declaration hides variable "err" (declared at line 3
 Line6 700)
 Line7          { int err = SSL_get_error(soap->ssl, r);
 Line8                ^

My output:

"/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp",
line
1657:
error
#4296-D:
arithmetic
operation
on
boolean
type
{
return
soap->count
-
soap->buflen
+
soap->bufidx
-
(soap->ahead
!=
0);
"/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp",
line
4136:
warning
#3348-D:
declaration
hides
variable
"err"
(declared
at
line
3
700)
{
int
err
=
SSL_get_error(soap->ssl,
r);
^

Thanks

`while loop takes much time to process` -- who told you that? — devnull, Jan 28 '14 at 09:27
awk works is line-base per default `awk '1' warnings.txt` will print each line. What is your real question? — Fredrik Pihl, Jan 28 '14 at 09:27
awk is the perfect tool do to something to each line, please update your question with what you would like to do and what the expected outcome is and we gladly help you! — Fredrik Pihl, Jan 28 '14 at 10:33
@user1502952, You made wrong decision base on wrong understanding. Show detail and explain to us what you really need to archive. — BMW, Jan 28 '14 at 10:50
NO, you just added line-numbers. Again, what would you like to do with each line? Extract the error, get the line-number? Create a LaTeX-table of the messages? We have no crystal-ball available so PLEASE tell us what you would like to do with each line? — Fredrik Pihl, Jan 28 '14 at 11:15
After your latest edit, will `awk '{for(i=1;i<=NF;i++)print $i}' warnings.txt` solve your problem? — Fredrik Pihl, Jan 28 '14 at 11:52
@user1502952 the problem is you're asking us how to do something that makes no sense to do. It's like asking us how to jump-start your car when it has a dead battery because you want to go for a bike ride. If you can just take a step back and THINK about what output you want to produce from reading warnings.txt and then post that output with an explanation of the process, we can help you. — Ed Morton, Jan 28 '14 at 15:48

score 3 · Answer 1 · answered Jan 28 '14 at 09:32

3

Use the read builtin instead

while read -r line; do
    echo "$line"
done < warning.txt

answered Jan 28 '14 at 09:32

helpermethod

59,493
71
188
276

i want to avoid using while loop, Please check my query – user1502952 Jan 28 '14 at 09:39
1

But your version using `awk` will be less portable (requiring an external command) and probably slower (because of the overhead of forking another process.) – helpermethod Jan 28 '14 at 09:41
Using awk will not be less portable, nor will it be slower. Using awk will, however, be more concise, more robust, and more easily extenisble and maintainable. – Ed Morton Jan 28 '14 at 15:16

score 1 · Answer 2 · answered Jan 28 '14 at 15:22

Here is how you use awk to read warnings.txt and re-produce the file as-is, per your posted question:

$ awk '1' warnings.txt
 Line1  "/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp", line 1657: error #4296-D: arithmetic operation on boolean type
 Line2{ return soap->count - soap->buflen + soap->bufidx - (soap->ahead != 0);
 Line3
 Line4
 Line5  "/Project/user/XYZ/gsoap/gsoap-2.8.9/gsoap/stdsoap2.cpp", line 4136: warning #3348-D: declaration hides variable "err" (declared at line 3
 Line6 700)
 Line7          { int err = SSL_get_error(soap->ssl, r);
 Line8                ^

Obviously you could do that with cat instead of awk. If you have anything else you want to do with the contents of that file, please do let us know (by updating your question with expected output and an explanation) so we can help you.

score -3 · Answer 3 · answered Jan 28 '14 at 10:23

-3

You could do that:

Code:

ifsold="$IFS"
IFS="
"
for line in `cat warnings.txt` ; do
    echo "line: $line"
done
IFS="$ifsold"

answered Jan 28 '14 at 10:23

Dominik Heidler

55
1
7

2

This is worse than `while` loop, which is allegedly too slow. Namely the `cat` is not needed, but it can also cause buffering issues and other problems. – Reinstate Monica Please Jan 28 '14 at 10:25
It it is at least addressing the `while` problems: It's faster and you have access to local vars as you are not starting a subprocess for each line. – Dominik Heidler Jan 28 '14 at 10:33
1

There are no `while` problems, `while` will always be faster than his syntax and at least never be slower than your syntax, and less error prone. `awk` by itself might be faster in a lot of cases, but that's a different question. – Reinstate Monica Please Jan 28 '14 at 10:37
3

Never write `for line in \`cat file\` ; do stuff; done`, write `while IFS= read -r line; do stuff; done < file` instead. – Ed Morton Jan 28 '14 at 15:18

Reading file line by line using awk and for loop

3 Answers3