1

I have data affixed with a time-stamp coming from my client and passed to my server via web sockets.

Sometimes the order will be wrong so that means the timestamps will be in the wrong order. So, I created a SortedDictionary collection object:

  public SortedDictionary<Int64, string> Manifest = new SortedDictionary<Int64, string>();

This data is coming thick and fast (as quickly as possible) and is real-time with as much as 10 data 'blocks' every second.

But, also, sometimes it can take up to 10 seconds for the next data 'block' to come through.

The other client that should recieve the correct order of this data stream needs to be real-time as well.

But, this real-time demand cannot be done as far as I can see so I thought about creating a cache of say 10 seconds worth of data.

Now, you would think that this would be easy but I have had errors such as:

'entry already added'
'cannot find key'

These errors seem to be derived from the fact you cannot modify this type of collection at the same time as reading it off.

My code is this so far:

In my receive event in my web socket I have this:

Manifest.Add(timestamp, 'data');

and in, say my timer (running every 100ms):

var pts = Manifest.ElementAt(0);
var key = res.TS;
//my 10 second cache
if (Convert.ToInt64(DateTime.Now.ToString("yyyyMMddHHmmssfff")) - key >= 10000)
{
    //send the data
    SendDataToClients(key .data);
    //remove send data from queue
    Manifest.Remove(res);
}

Now, I am sure people will say the problem is that I am adding to this collection at the same time as I am reading it and a way to solve this is to use a lock.

But I am loathed to use a lock because it can create an overhead and as I am dealing in real-time( albeit with a second lag) I cannot afford ANY overhead.

Are they any other approaches? I have working on this all day and have tried many many things.

The above was my best approach.

Andrew Simpson
  • 6,883
  • 11
  • 79
  • 179
  • 2
    Why are you loathed to use locking, can't imagine it will result in a delay anywhere near the 10 second cache you have currently. As for your dictionary issue, have you seen this approach? http://stackoverflow.com/questions/22308067/thread-safe-sorteddictionary – timothyclifford Jul 20 '15 at 15:56
  • 1
    10 seconds delay and "can't afford single lock on updating collection" does not sound reasonable together... Consider clarifying your time requirements. – Alexei Levenkov Jul 20 '15 at 15:56
  • @timothyclifford Hi, I have to leave 10 seconds so I can have time to sort the data out. I would have to use the locking as well as the cache you see. How would a dictionary help me? I am using the SortedDictionary to automatically sort the timestamps out - but thanks for your comment – Andrew Simpson Jul 20 '15 at 15:59
  • You can use `concurrentdictionary` if you don't want to use `lock` – AD.Net Jul 20 '15 at 16:00
  • @AlexeiLevenkov Hi, thanks for your comment. Yes, I agree I cannot see it working. Hence my post here. I WILL be considering my time requirements if I cannot find any weird inspiration from anywhere :) – Andrew Simpson Jul 20 '15 at 16:00
  • @AD.Net Sounds promising. I shall take a look thanks. Chaps I have been called into a meeting so cannot answer to anything for a short while :( – Andrew Simpson Jul 20 '15 at 16:01
  • @timothyclifford Hi, sorry I did not realise you had given me a link. thank you – Andrew Simpson Jul 20 '15 at 17:33
  • No problems, hope it helps :) – timothyclifford Jul 20 '15 at 20:28

0 Answers0