7

I'm measuring some system performance data to store it in a database. From those data points I'm drawing line graphs over time. In their nature, those data points are a bit noisy, ie. every single point deviates at least a bit from the local mean value. When drawing the line graph straight from one point to the next, it produces jagged graphs. At a large time scale like > 10 data points per pixel, this noise is compressed into a wide jagged line area that is, say, 20px high instead of 1px as in smaller scales.

I've read about line smoothing, anti-aliasing, simplifying and all these things. But everything I've found seems to be about something else.

I don't need anti-aliasing, .NET already does that for me when drawing the line on the screen.

I don't want simplification. I need the extreme values to remain visible, at least most of them.

I think it goes in the direction of spline curves but I couldn't find much example images to evaluate whether the described thing is what I want. I did find a highly scientific book at Google Books though, full of half-page long formulas, which I wasn't like reading through now...

To give you an example, just look at Linux/Gnome's system monitor application. I draws the recent CPU/memory/network usage with a smoothed line. This may be a bit oversimplified, but I'd give it a try and see if I can tweak it.

I'd prefer C# code but algorithms or code in other languages is fine, too, as long as I can port it to C# without external references.

ygoe
  • 18,655
  • 23
  • 113
  • 210
  • Are you aware that windows (and .net) have a built-in performance counter system? I just want to make sure you're not reinventing the wheel (not saying that they necessarily apply to your case). – Klaus Byskov Pedersen Dec 08 '10 at 15:07
  • My data is collected on a Linux server and consists of a multitude of different sources that I have other code for. The data shall be visualised for use in a desktop (interactive) or web application or possibly e-mailed reports under the Mono runtime. Development is done with Visual Studio on Windows. – ygoe Dec 22 '10 at 16:47

5 Answers5

6

You can do some data-smoothing. Instead of using the real data, apply a simple smoothing algorithm that keeps the peaks like a Savitzky-Golayfilter.

You can get the coefficients here.

The easiest to do is:

Take the top coefficients from the website I linked to:

// For np = 5 = 5 data points
var h = 35.0;
var coeff = new float[] { 17, 12, -3 }; // coefficients from the site
var easyCoeff = new float[] {-3, 12, 17, 12, -3}; // Its symmetrical
var center = 2; // = the center of the easyCoeff array

// now for every point from your data you calculate a smoothed point:

smoothed[x] = 
   ((data[x - 2] * easyCoeff[center - 2]) +
    (data[x - 1] * easyCoeff[center - 1]) +
    (data[x - 0] * easyCoeff[center - 0]) +
    (data[x + 1] * easyCoeff[center + 1]) +
    (data[x + 2] * easyCoeff[center + 2])) / h;

The first 2 and last 2 points you cannoth smooth when using 5 points.

If you want your data to be more "smoothed" you can experiment with coefficents with larger data points.

Now you can draw a line through your "smoothed" data. The larger your np = number of points, the smoother your data. But you also loose peak accuracy, but not as much when simply averaging some points together.

GvS
  • 52,015
  • 16
  • 101
  • 139
  • 1
    I have implemented a variant of this now. First I am averaging the raw source data points down to ~3 values per pixel. This produces a more uniform data density when rendering months of data with varying resolution from seconds to minutes. I then run those values through the largest coefficient list that can be found on the other site. It indeed produces more smoothed lines in the graph. But very small and extreme peaks produce funny effects in the graph. It oscillates greatly in the other direction around the curve where the extreme is. I think it comes from the negative coefficients. – ygoe Dec 22 '10 at 18:16
2

You cannot fix this in the graphics code. If your data is noisy then the graph is going to be noisy as well, no matter what kind of line smoothing algorithm you use. You'll need to filter the data first. Create a second data set with points that are interpolated from the original data. A Least Squares fit is a common technique. Averaging is simple to implement but tends to hide extremes.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536
1

I think what you are looking for is a routine to provide 'splines'. Here is a link describing splines:

http://en.wikipedia.org/wiki/Spline_(mathematics)

If that is the case I don't have any recommendations for a spline library, but an initial google search turned up a bunch.

Sorry for no code, but hopefully knowing the terminology will aid you in your search.

Bob

BlueRaja - Danny Pflughoeft
  • 84,206
  • 33
  • 197
  • 283
rcravens
  • 8,320
  • 2
  • 33
  • 26
0

Reduce the number of data points, using MIN/MAX/AVG before you display them. It'll look nicer and it'll be faster

smirkingman
  • 6,167
  • 4
  • 34
  • 47
0

Graphs of network traffic often use a weighted average. You can sample once per second into a circular list of length 10 and for the graph, at each sample, graph the average of the samples.

If 10 isn't enough you can store many more. You don't need to recalculate the average from scratch, either:

new_average = (old_average*10 - replaced_sample + new_sample)/10

If you don't want to store all 10, however, you can approximate with this:

new_average = old_average*9/10 + new_sample/10

Lots of routers use this to save on storage. This ramps toward the current traffic rate exponentially.

If you do implement this, do something like this:

new_average = old_average*min(9,number_of_samples)/10 + new_sample/10
number_of_samples++

to avoid the initial ramp-up. You should also adjust the 9/10, 1/10 ratio to actually reflect the time preiod of each sample because your timer won't fire exactly once per second.

Eyal
  • 5,728
  • 7
  • 43
  • 70