1

upd: Let me rephrase my question shortly. There are N double numbers. There are N dedicated threads each of them update own double number (_cachedProduct in the example below).

Somehow I need to have sum of these numbers and I need IndexUpdated event to be raised ASAP after any double number is changed (it would be nice if such event can be raised in 10 µs or less).

Below is how I tried to implement this task

===============================================

To calculate stock exchange index I create private double[] _cachedProduct; field. These field is written by many threads

    // called from another threads
    public override void InstrumentUpdated(Instrument instrument)
    {
        if (!_initialized)
        {
            if (!Initialize())
            {
                return;
            }
        }
        int instrumentId = instrument.Id;
        OrderBook ob = Program.market.OrderBook(instrument);
        if (ob.MedianOrAskOrBid == null)
        {
            _cachedProduct[instrumentId] = 0;
        }
        else
        {
            _cachedProduct[instrumentId] = ((double) ob.MedianOrAskOrBid)*_ammounts[instrumentId];
        }
    }

_ammounts is pre-initialized array and please ignore Initialize method and variable - they just works.

In loop I just sum all _cachedProduct and when values changes I notify others.

        Task.Factory.StartNew(() =>
                {
                    while(true)
                    {
                        if (_initialized)
                        {
                            break;
                        }
                    }
                    while (true)
                    {
                        CalculateAndNotify();
                        //Thread.Sleep(5);
                    }
                }
            , TaskCreationOptions.LongRunning);


    protected void CalculateAndNotify()
    {
        var oldValue = Value;
        Calculate();
        if (oldValue != Value)
        {
            NotifyIndexChanged();
        } 
    }

    protected override void Calculate()
    {
        double result = 0;
        for (int i = 0; i < _instrumentIds.Count(); i++)
        {
            int instrumentId = _instrumentIds[i];
            if (_cachedProduct[instrumentId] == 0)
            {
                Value = null;
                return;
            }
            result += _cachedProduct[instrumentId];;
        }
        Value = result;
    }

I must use Interlocked to update my double _cachedProduct values but please ignore that fact now, what other problems with this code do you see?

Should I call Calculate method inside while(true) so I always use one core without delays. My machine has 24 cores so I was thinking this is ok.

However without Thread.Sleep(5) (commented) I do see significant slow-down in the program overall and I do not understand why. Program executes several dozens times slower in many places.

The question is if my idea of using while(true) without any locking at all is OK. Or should I introduce some locking method so I would only Calculate index when one of of _cachedProduct is updated?

Oleg Vazhnev
  • 23,239
  • 54
  • 171
  • 305
  • explain what you mean by significant slowdown please. – M Afifi Jul 09 '12 at 09:23
  • somehow program post much less orders and loads orders table from server much much slower (10-100 times slower) and I do not understand why taking into account that in code that changed I do not use any locks... – Oleg Vazhnev Jul 09 '12 at 09:25
  • Do you really need to pull for change notifications? isnt this much slower with a high number of products compared to just pushing notifications. When pulling in a tight loop, you're consuming alot of memory bandwith. – Polity Jul 09 '12 at 09:28
  • @Polity I don't understand your question. The scenario is like that: assume DJ is trading by 13 000 and INTC is trading by 23. In the moment DJ is changed to 12 900 but INTC is still trading by 23. I attach INTC strategy to be notified about DJ changes and when it sees that INTC is overpriced it sell it. As you understand delays in such strategies are not acceptable. – Oleg Vazhnev Jul 09 '12 at 09:32
  • @javapowered - I understand but by having a single thread pull on all instances of an array, you're preventing any CPU optimization on that array and its instances. Meanwhile you keep on loading that same array from memory over and over which is not really good for performance since memory bandwith is limited. Another strategy would be to make a thread look at a blocking collection and each time a change is made to a product, you push a notification in that blocking collection. – Polity Jul 09 '12 at 09:38
  • @Polity as I understand from other HFT guys `BlockingCollections` are extremely slow for HFT and should be avoided. Some `lock-free` techniques should be used, but `BlockingCollection` uses "true" locking inside – Oleg Vazhnev Jul 09 '12 at 10:27
  • @javapowered - You're assumption that no locking is 'always' faster than locking is false. BlockingCollection might use locking which uses a semaphore in high performant scenarios. (I say might because it depends on the internal producer-consumer collection. Using a concurrentQueue for example does not use locking inside). Anyways, my point is that looping over an array, therefore consuming memory bandwith, might very well be way more expensive than simply locking and working with the first item of a queue – Polity Jul 10 '12 at 02:30

3 Answers3

1

I think you might get better performance and clearer code if you do not use an extra thread and loop for your sum. On every change to an instrument you calculate the difference and immediately update the index and perform the notify

So if a thread calls InstrumentUpdated for a single instrument;

  change = newvalue - currentvalue;
  // used interlocked here to change the index threadsafe
  StockExchangeSum = Interlocked.Add(ref StockExchangeSum,change);
  NotifyIndexChanged();
IvoTops
  • 3,463
  • 17
  • 18
  • I was thinking about that initially, but I'm afraid that this will not work cause after many millions updates I will not have "true" StockExchangeSum because `double` is not `exact` number and always has "relative error" If on every step I will add, say 10^10 error than soon I will have completely wrong value. May be i'm wrong I probably should test that. – Oleg Vazhnev Jul 09 '12 at 10:30
  • 1
    For money values and stock prices I always use a decimal which is meant for this, see http://msdn.microsoft.com/en-us/library/364x0z75(v=vs.80).aspx – IvoTops Jul 09 '12 at 13:54
  • He said performance critical, Decimal won't cut it, http://stackoverflow.com/questions/366852/c-sharp-decimal-datatype-performance – M Afifi Jul 09 '12 at 14:34
  • 2
    I think the threadsafety will cost him more performance as the difference between double and decimal. For decimals I can do 18 million c+=(b-a) per second single threaded. I work daily with streams of quotes in decimals and my profiler tells me that's not it ;-) – IvoTops Jul 10 '12 at 09:49
  • @IvoTops there are no `Interlocked.Add` for decimal isn't it? or how can I convert `decimal` to `Int64`? – Oleg Vazhnev Jul 17 '12 at 19:40
  • @javapowered If you need to hang on to Int64 for the Interlocked you can use Int64Value = (long) (DecimalValue * fixdecimals) where you set fixdecimals = 3 to get a decimal 123.456 into the Int64 asa value 123456 – IvoTops Jul 18 '12 at 09:06
0

Can double[] be a more complex type? How does WaitHandle.WaitAny compare performance wise?

Something like as follows.

private Index[] indicies;

public class Index
{
    public WaitHandle Updated =
        new EventWaitHandle(false, EventResetMode.AutoReset);
    public double _value;
    public double Value
    {
        get {return _value;}
        set
        {
            if(_value != value)
            {
                _value = value;
                Updated.Set();
            }
        }
    }
}

TaskFactory.StartNew(() =>
{
    while(true)
    {
        WaitHandle.Any(indicies.Select(i => i.Updated));
        CalculateAndNotify();
    }
});
M Afifi
  • 4,645
  • 2
  • 28
  • 48
0

Some points for you to think about

  • Have you tried profiling your calculation block in isolation to the rest of the code? I noticed this in your Calculate function:

    for (int i = 0; i < _instrumentIds.Count(); i++)

    _instrumentIds.Count() invokes an iteration over the entire collection and it is possible this is invoked for each trip around the loop. i.e. you are doing N^2/2 iterations of _instrumentIds

  • Is the _instrumentIdsIEnumerable being modified during this calculation operation? If so you could get all sorts of race conditions leading to incorrect answers.

  • Is the Task containing CalculateAndNotify called once or is it called many times (nested)? E.g. is there some operation inside CalculateAndNotify that could cause it to be triggered recursively?

    If so, you might find you have several calculations performing simultaneously (using more than one thread until the pool is starved). Can you include some logging on start/end of operation and perhaps count the number of simultaneous calculations to check this?

    If this is an issue you could include some logic whereby the CalculateAndNotify operation is queued up and new calculate operations cannot be executed until the previous has completed.

Dr. Andrew Burnett-Thompson
  • 20,980
  • 8
  • 88
  • 178
  • `CalculateAndNotify` is called from only one place and so can not be executed simultaneously – Oleg Vazhnev Jul 09 '12 at 13:21
  • Ok, so my other questions - have you profiled this block in isolation to the rest of the code? What about the count()? Finally I notice that you are calling CalculateAndNotify in a Task with TaskCreationOptions.LongRunning set. Try creating a dedicated thread and setting the priority lower. Also see this related question which discusses performance of Threads vs. Tasks vs. Threadpool and how Task throttling can adversely affect your application http://stackoverflow.com/questions/8676854/why-there-is-so-much-performance-diffrence-between-tasks-thread-and-threadpool – Dr. Andrew Burnett-Thompson Jul 09 '12 at 14:18
  • @Dr.ABT .Count() will use .Count (the property) if available. As he's using an indexer on it (IList), then it is an ICollection too and .Count is available. .Count() is a red herring (although using the property is obviously more efficient but branch prediction will remove the vast majority of the performance overhead of the method call). – M Afifi Jul 09 '12 at 14:43