How to compute spectrum using awk or shell scripting?

Question

I would like to compute spectrum using awk or shell scripting. I have a data, e.g.,

ifile.txt
1
2
3
4
1
3
2
2
3
99

Where 99 is an undefined value.

Formula to compute spectrum is for k=1,2,3,4,...

I was doing it in the following way.

for i in {1..10};do
awk '{if($1 != 99) printf "%f %f\n", 
     $1*sin(2*3.14*'$i'*NR/10), 
     $1*cos(2*3.14*'$i'*NR/10)}' ifile.txt > ifile1.txt
sum_1=$(awk '{sum += $1} END {print sum}' ifile1.txt)
sum_2=$(awk '{sum += $2} END {print sum}' ifile1.txt)
awk '{printf "%f\n", (1/2)*(((1/5)*('$sum_1')^2)+((1/5)*('$sum_2')^2))}' 
     >> ofile.txt
done

Would please suggest where I am doing mistake. The computation is neither printing anything nor ending. However I am getting values in ifile1.txt

First, trying to pass shell variables to awk via interpolation is a bad idea; and second, it won't work at all with single-quoted string; third, that is not an awk program. — miken32, Mar 10 '17 at 04:42
both awk and bash are horrible programming languages. you should do this in a scripting language, perl/python/ruby/javascript to make it readable and maintainable. — Andy Ray, Mar 10 '17 at 05:13
@AndyRay you sound like someone with extremely limited knowledge of awk or bash and a fundamental misunderstanding of what a scripting language is. I'm not saying awk or bash are the right tools **for this job** but they are both absolutely excellent at what they do (and the languages used by both tools are scripting languages). Kay - you said you are `getting errors` - you stand a much better chance of getting help debugging those errors if you tell us what the error messages are. General advice - start with a simpler formula and get the syntax for THAT right, then add complexity. — Ed Morton, Mar 10 '17 at 06:06
@Kay The `END` inside the formulas does confuse me. AFAIK `END` is a special pattern for actions which are executed when all input has been processed. I do not know whether it may even be used inside of actions and how awk reacts on this. Please, edit your question and add the error output. (as already recommended by Ed Morton) — Scheff's Cat, Mar 10 '17 at 07:02
@Kay 2nd last line wrong. It must be: `'{printf "%f\n", (1/2)*(((1/5)*('$sum_1')^2)+((1/5)*('$sum_2')^2))}'`. You cannot use shell variables in awk scripts but you can embed them by shell. Hence the quotes around the shell variables... (It's like with `$i`.) — Scheff's Cat, Mar 10 '17 at 09:07
@Scheff .Thank you. But still it is not printing anytthing. The script is also not terminating. — Kay, Mar 10 '17 at 09:11
@Kay The last awk line expects input. You have to give end of input (`^D` I believe). — Scheff's Cat, Mar 10 '17 at 09:14
@Kay This should work: `awk 'BEGIN {printf "%f\n", (1/2)*(((1/5)*('$sum_1')^2)+((1/5)*('$sum_2')^2))}' /dev/null`. `BEGIN` is important - otherwise the script does nothing because `/dev/null` means read (from) nothing. — Scheff's Cat, Mar 10 '17 at 09:17

Scheff's Cat · Accepted Answer · 2017-03-10T09:40:25.060

After our successful dialog I compiled our common effort into a self-standing awk script (spectrum.awk):

$1 < 99 {
  for (i = 1; i <= 10; ++i) {
    sum1[i] += $1*sin(2*3.14*i*NR/10)
    sum2[i] += $1*cos(2*3.14*i*NR/10)
  }
  next
}
END {
  for (i = 1; i <= 10; ++i) {
    printf "%f\n", (1/2)*(((1/5)*(sum1[i])^2)+((1/5)*(sum2[i])^2))
  }
}

It uses arrays (sum1 and sum2) to compute all 10 values in one run.

Unfortunately, I don't know anything about the theoretical background. I cannot see your image (due to proxy issues of my company). Thus, you may give feedback if the computation is wrong.

Sample session:

$ echo '1
2
3
4
1
3
2
2
3
99' | awk -f spectrum.awk
1.417046
2.019819
0.288438
2.688501
0.100023
2.680672
0.296974
1.993613
1.338068
44.097319

At least, it looks "nice".

Dear all, thank you very much for your suggestions and help. I could go few steps up. However, still not solved. Would you please see my updates. — Kay, Mar 10 '17 at 09:03

How to compute spectrum using awk or shell scripting?

1 Answers1