I have a small project which needs to track stock price % movement for specified time intervals. E.g. It should be able to track AAPL price % move for the last 30sec, 1min, 3min, 5min, 10min. All stocks share the same time intervals. Program will be written in C++ and will be running on Linux, though it shouldn't matter.
Below is the approach I am thinking of,
1) Get stock market data from multicast feed, for around 4000 symbols
2) Process the feed and store them in a global quotes cache
3) Spawn N number of threads, one for each interval
4) Each thread will maintain its own copy of symbols, on startup initialize each symbol with current price from the global cache, then go to sleep for specified interval
5) On wake up, snap the current price (from the quote cache which would have the latest value from the quotes feed), calculate the difference, push this information to a queue
6) Have a dedicated thread to read the queue and publish the prices to a caching server
7) Front end GUI applications read from the caching server and display the percent moves
This approach should work, but is there a better way to do this? Also one area of optimization could be coalescing the thread price snapping part(after the thread wakes up), where if two or more threads will be snapping at the same time, e.g. the 30sec, 1min and 2min threads will snap together every two minutes, so combine them into one snap. What logic can I use to do the coalescing? Thank you.