1

I have "online" series of integers number, so every several milliseconds I have new number. Frequency is not specified - sometimes I have many numbers, sometimes I have no numbers.

In reality this numbers are so called "true price" of the stock. I use it in my trading application.

Now I just use last number as a "true price" so I do not track series at all. But there are some problems with such approach. Let's look at this series:

  1. 98; last = 98
  2. 100; last = 100
  3. 101; last = 101
  4. 100; last = 100
  5. 101; last = 101
  6. 100; last = 100
  7. 101; last = 101
  8. 100; last = 100
  9. 101; last = 101
  10. 100; last = 100
  11. 99; last = 99
  12. 98; last = 98

The problem is that in very short period of time my "true price" is changed from 100 to 101 and back too many times. Every change of true price implies a lot of work (recalculations, orders placement etc) so i do not need to change true price like that. This change (100-101-100-101) is due "measurement problems" and i need to filter it. I can not fix "measurement" so I have to fix it here.

One of the approach would be to use "3LastAverage" algorithm:

  1. 98; 3lastAverage = 98
  2. 100; 3lastAverage = 99
  3. 101; 3lastAverage = 99.67 => 100
  4. 100; 3lastAverage = 100.33 => 100
  5. 101; 3lastAverage = 101
  6. 100; 3lastAverage = 100
  7. 101; 3lastAverage = 101
  8. 100; 3lastAverage = 100
  9. 101; 3lastAverage = 101
  10. 100; 3lastAverage = 100
  11. 99; 3lastAverage = 100
  12. 98; 3lastAverage = 99

As you can see such approach doesn't always work, i still have 100-101-100 problem. Probably i can use more numbers to calculate "average"... but such approach will not work for me because of such example:

  1. 99; 3lastAverage = 99 (i want 99)
  2. 100; 3lastAverage = 100 (i want 100)
  3. 101; 3lastAverage = 100 (i want 101)
  4. 102; 3lastAverage = 101 (i want 102)
  5. 103; 3lastAverage = 103 (i want 103)

In general, when things go "OK" i need truePrice to be just the last number!

So, having "online" series of integers numbers, i need to calculate so called "true value" of this series, which defined this way:

  • if everything is OK then this is just the last number
  • if there are too much the same number in series (because measurement problems) then "true value" shouldn't change too often. most adequate value should be used

My suggestion would be:

  • just prohibit "true value" to have the same price more than once per second.

For example:

  1. 0.000 : 98; true value = 98
  2. 0.100 : 100; true value = 100
  3. 0.200 : 101; true value = 101
  4. 0.300 : 100; true value = 101 (100 was already used at step 2 0.2 seconds ago)
  5. 0.400 : 98; true value = 101 (98 was already used at step 1 0.4 seconds ago)
  6. 0.500 : 99; true value = 99
  7. 0.600 : 100; true value = 99 (100 was used at step 2 0.5 seconds ago)
  8. 1.500 : 101; true value = 101 (more than one second is elapsed since 101 was used).

However such approach also has such "bug":

  1. 99 tp = 99
  2. 100 tp = 100
  3. 101 tp = 101
  4. 102 tp = 102
  5. 103 tp = 103
  6. 102 tp = 103
  7. 101 tp = 103
  8. 100 tp = 103
  9. 99 tp = 103
  10. 98 tp = 98
  11. 97 tp = 97

The problem here is that "tp" was "frozen" at level 103 for too long time.

I'm really sorry for such a big question. But probably someone was solving close problem and can share experience. My primary problem is that I do need to solve two opposite problems at the same time:

  • "true price" must just be the "last" value under general conditions
  • "true price" shouldn't change to often (so I need to use previous values one way or another)
  • also it's hard to say when we have "general" conditions and when we have "measurement problems"

So my question is really vague and I just hope that someone was tring to solve something like that. As always "common sense" should be used solving this problem.

Oleg Vazhnev
  • 23,239
  • 54
  • 171
  • 305
  • difficult question, very vague... anyway: I don't understand if this is an applicaiton with some users. In this case, does it make snese to update the value each second?+ – Daniele B Dec 21 '12 at 12:57
  • 1
    Isn't it possible like if some price (say 101) is reached often, then store the corresponding computation results in a cache memory (with a certain time-out option). So next time a value is reached again, you dont have to recompute. – Gaurav Dec 21 '12 at 13:02
  • @Gaurav no, I just really want to make "true price" to be a series that doesn't change too often. havinch such assumption is very handy in underlying code. also it's pretty natural - i don't want measurements problems to break my "true price". – Oleg Vazhnev Dec 21 '12 at 13:32
  • 1
    I don't think this is really a programming question at all. OP has to decide what to implement and then get on with it. The answer to this question will not be found in any reference on algorithms or programming, but perhaps in a reference on algorithmic trading. – High Performance Mark Dec 21 '12 at 13:32

1 Answers1

5

You want to respond to long-term (low frequency) changes in price, but ignore small short-term (high frequency) fluctuations. So what you need is a low-pass filter.

There are lots of different kinds of low-pass filters with different characteristics. A moving average is one kind of low-pass filter, but it has the problem (which you observe) that it trades smoothing for lag (in order to smooth out high-frequency fluctuations you need to average over quite a long sequence, but then it takes a long time for low-frequency changes to become apparent).

So you might try out the other low-pass filters, for example the exponentially-weighted moving average, and see if they meet your requirements.

The other thing to say about this task is that basing your decision about when you do a recalculation on the nearest integer to your moving average is a bad thing to do, because it amplifies small fluctuations when the moving average is close to halfway between two integers. (If the moving average goes from 99.49 to 99.51 the rounding operation amplifies this small fluctuation to a big one from 99 to 100.)

What you should do instead is use hysteresis to avoid this kind of response. When you do a recalculation, record the current value a0 of your moving average a, and don't do another recalculation until the moving average a has moved by at least ε, that is, |a − a0| ≥ ε.

Gareth Rees
  • 64,967
  • 9
  • 133
  • 163
  • thanks, the question is then which low-pass filter is better for me :) i really want to have just the "last number" in most cases when there are no noise. – Oleg Vazhnev Dec 22 '12 at 09:03