How to calculate extra large numbers with bc in bash

Question

I have a file with 1800 lines that look like this

600.76
600.66
700.44
566.66
Ect..

I made a bash script to calculate the mean.

Now I first made a variable to count the total column lines like:

Lines="$(awk 'END{print NR}' file.txt)"

Then another variable for the sum of that column like this:

Sum="$(awk '{s+1=$1}END {print s}' file.txt)"

Lastly I'm finding the mean like this:

Echo "scale=2 ; $Sum / $Lines" | bc

With debugging enabled It returns:

+echo 'scale=2 ; 1.72161e+06 / 1800'
(Standard_1): syntax error

I realize now bc doesn't do scientific notation but how do I get around this.

I'm OK with short handing the decimal by restricting it to 2 or 3 places.

@twallberg awk returns scientific notation after the answer gets too large. Which is the original problem — theloosegoos, Jan 31 '17 at 19:48
@theloosegoos The perhaps `awk '{sum+=$1}END{printf "%f\n", sum/NR}` would provide better results. `awk` has a `printf` implementation that mimics a lot of (but not all) the C version features... — twalberg, Jan 31 '17 at 21:51
@theloosegoos Are any lines in the input file in scientific notation? — Ian Petts, Feb 01 '17 at 02:41
No I'm saying the file only has one column and it's a number like this 857.54. There are 1800 lines simular to that. When you sum the file the out put is in scientific notation. There for when I attempt to pass that to bc for some division it tosses the error. — theloosegoos, Feb 01 '17 at 02:49

gudok · Answer 1 · 2017-02-01T07:23:23.327

0

Use bc -l for both summation and final division:

sum=0
count=0
while read number; do
  number=$(printf "%f\n" $number) # get rid of scientific notation
  sum=$(echo "$sum" '+' "$number" | bc -l)
  count=$((count + 1))
done < input
avg=$(echo $sum / $count | bc -l)
echo $avg

edited Feb 01 '17 at 07:23

answered Jan 31 '17 at 18:57

gudok

4,029
2
20
30

This will lead me to the same error because the sum of the file is to large it's expressed in scientific notation. – theloosegoos Jan 31 '17 at 20:09
@theloosegoos No, it won't. `bc` doesn't use scientific notation. It is an arbitrary precision calculator. – gudok Jan 31 '17 at 20:24
the root of the problem is when the column of my file is added. The sum is so large it's expressed in scientific notation. So when I pass the the sum of the column to bc it will toss the error because it doesn't accept the notation. – theloosegoos Jan 31 '17 at 21:28
@theloosegoos, can you provide the input file? – gudok Jan 31 '17 at 21:35
its the very first example in the post given. Aside from its just the first of 1800 lines – theloosegoos Jan 31 '17 at 22:19
@theloosegoos, bc never uses scientific notation. If you still get an error, then I guess that one of the input numbers is already represented in scientific notation. You may use `printf` to convert number back to ordinary notation. I've updated code in my post. – gudok Feb 01 '17 at 07:25

score 0 · Answer 2 · answered Jan 31 '17 at 19:01

0

Unnecessary to use awk. Simple oneliner can do the job.

echo "scale=2; ("$(paste -sd+ file.txt)")"/$(wc -l <file.txt)|bc

answered Jan 31 '17 at 19:01

Ipor Sircer

3,069
3
10
15

This still sends bc into a error it can't handle scientific notation this happens whenext the answer goes above a certain value. – theloosegoos Jan 31 '17 at 19:28

How to calculate extra large numbers with bc in bash

2 Answers2