High-frequency data real-time curve display through database , is it feasible?

Question

Monitoring project, 16 sensors, sampling frequency 50hz, oracle database is adopted, with simple structure: record_time + sensor_data.

Create Table real_data(
    record_time timestamp(3),
    ac_1 Float,
    ac_2 Float,
    ac_3 Float,
    ac_4 Float,
    ac_5 Float,
    ac_6 Float,
    ac_7 Float,
    ac_8 Float,
    ac_9 Float,
    ac_10 Float,
    ac_11 Float,
    ac_12 Float,
    ac_13 Float,
    ac_14 Float,
    ac_15 Float,
    ac_16 Float
    )
    Tablespace data_test;

I uses the livecharts wpf control to read the database, displaying real-time curves.

Requirements: 20ms display a data, curve shift left, no displaying pause.

Now two ways to read the database:

1. Regular refresh

   Timer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(1000) };

The problem is that the interval of 1 second, read the latest 1 second data in the database (about 50 data), the curve has a pause (shift left every 1 second); transferred to 20ms refresh, read the latest data and insert after the curve, many times to read the same data, because the database query time in 100ms or so (select top ), resulting in a lot of straight line curve, does not meet the actual trend of change.

2. The use of thread, specifically a thread to read the database, select the starting time, there is occasional pause, the reasons for the analysis: the time to read the data is generally 20ms, and occasionally 300ms, Because the database there are frequent insert and delete, while the operating system memory is also real-time changes, will affect the speed of reading. And the longer the time, the more severe the curve delay.

Whether high frequency data display, through the database to read the way is not feasible? Only through the direct read device API feasible?

Thank you!

Answer 1

You are asking, how to refresh a chart quicker than you can query your database, without pausing. I am no technical guru, but I know of a "cheat" that might be useful to you. Instead of showing the current values on the chart, show the values from a second ago, basically buffering.

So the idea is that you have a thread that reads data every 500ms, and store the read data into memory. Then you have another thread for chart update that takes one result set from memory every 20ms and draws it. That will make the chart looks to be smooth and without pause, also the data curve will be accurate. The only draw back is that the chart is not showing "live" data but a delayed version of the data from 500ms ago.

here's what i meant:

0 sec - sensor begins, no visual on chart

0.5 sec - 25 data set in DB, query DB (select 1~25), no visual on chart

1 sec - 50 data set in DB, query DB (select 26~50), starts displaying result 1~25 on chart over the next 500ms.

1.5 sec - 75 data set in DB, query DB (select 51~75), starts displaying result 26~50 on chart over the next 500ms.