2

Given input signal x (e.g. a voltage, sampled thousand times per second couple of minutes long), I'd like to calculate e.g.

/ this is not q
y[3] = -3*x[0] - x[1] + x[2] + 3*x[3]
y[4] = -3*x[1] - x[2] + x[3] + 3*x[4]
. . .

I'm aiming for variable window length and weight coefficients. How can I do it in q? I'm aware of mavg and signal processing in q and moving sum qidiom

In the DSP world it's called applying filter kernel by doing convolution. Weight coefficients define the kernel, which makes a high- or low-pass filter. The example above calculates the slope from last four points, placing the straight line via least squares method.

Xpector
  • 639
  • 1
  • 5
  • 17

3 Answers3

3

Something like this would work for parameterisable coefficients:

q)x:10+sums -1+1000?2f
q)f:{sum x*til[count x]xprev\:y}
q)f[3 1 -1 -3] x
0n 0n 0n -2.385585 1.423811 2.771659 2.065391 -0.951051 -1.323334 -0.8614857 ..

Specific cases can be made a bit faster (running 0 xprev is not the best thing)

q)g:{prev[deltas x]+3*x-3 xprev x}
q)g[x]~f[3 1 -1 -3]x
1b
q)\t:100000 f[3 1 1 -3] x
4612
q)\t:100000 g x
1791

There's a kx white paper of signal processing in q if this area interests you: https://code.kx.com/q/wp/signal-processing/

Fiona Morgan
  • 510
  • 2
  • 4
1

This may be a bit old but I thought I'd weigh in. There is a paper I wrote last year on signal processing that may be of some value. Working purely within KDB, dependent on the signal sizes you are using, you will see much better performance with a FFT based convolution between the kernel/window and the signal.

However, I've only written up a simple radix-2 FFT, although in my github repo I do have the untested work for a more flexible Bluestein algorithm which will allow for more variable signal length. https://github.com/callumjbiggs/q-signals/blob/master/signal.q

If you wish to go down the path of performing a full manual convolution by a moving sum, then the best method would be to break it up into blocks equal to the kernel/window size (which was based on some work Arthur W did many years ago)

q)vec:10000?100.0
q)weights:30?1.0
q)wsize:count weights
q)(weights$(((wsize-1)#0.0),vec)til[wsize]+) each til count v
32.5931 75.54583 100.4159 124.0514 105.3138 117.532 179.2236 200.5387 232.168.
Callum Biggs
  • 1,539
  • 5
  • 13
0

If your input list not big then you could use the technique mentioned here: https://code.kx.com/q/cookbook/programming-idioms/#how-do-i-apply-a-function-to-a-sequence-sliding-window

That uses 'scan' adverb. As that process creates multiple lists which might be inefficient for big lists.

Other solution using scan is:

 q)f:{sum y*next\[z;x]} / x-input list, y-weights, z-window size-1
 q)f[x;-3 -1 1 3;3]

This function also creates multiple lists so again might not be very efficient for big lists.

Other option is to use indices to fetch target items from the input list and perform the calculation. This will operate only on input list.

 q) f:{[l;w;i]sum w*l i+til 4} / w- weight, l- input list, i-current index
 q) f[x;-3 -1 1 3]@'til count x

This is a very basic function. You can add more variables to it as per your requirements.

Rahul
  • 3,914
  • 1
  • 14
  • 25