0

I have a real-time plotting system that retrieves and preprocess data from a stream and plots it, with each channel (from 1 to e.g. 100) plotted on a different 'row'. The version with pyqtgraph backend is complete and looks like:

pyqtgraph version

But it's not very performant. Thus, I wanted to use vispy, based on this example. @djhoese suggested I use the higher-level scenes instead, but I did not find a way to pass the entire array of data (channels, samples) at once as I do with the low-level gloo, and thus limiting the OpenGL call. As performance is my main reason for implementing the vispy backend for my real-time viewer, it didn't make sense to move away from the gloo version. Moreover, this version offers the possibility to set the number of rows and columns. For now, I use only one column, but one of the future improvements will be to select the number of rows/columns based on the number of channels. e.g. with a system using 128 channels, a 2-column window would be ideal.

I am slowly changing the original example code to match my need. Below, you can find the modified shaders (full-code here):

VERT_SHADER = """
#version 120
// y coordinate of the position.
attribute float a_position;
// row, col, and time index.
attribute vec3 a_index;
varying vec3 v_index;
// 2D scaling factor (zooming).
uniform vec2 u_scale;
// Size of the table.
uniform vec2 u_size;
// Number of samples per signal.
uniform float u_n;
// Color.
attribute vec3 a_color;
varying vec4 v_color;
void main() {
    float nrows = u_size.x;
    float ncols = u_size.y;
    // Compute the x coordinate from the time index.
    float x = -0.9 + 1.9*a_index.z / (u_n-1);
    vec2 position = vec2(x - (1 - 1 / u_scale.x), a_position);
    // Find the affine transformation for the subplots.
    vec2 a = vec2(1./ncols, 1./nrows);
    vec2 b = vec2(-1 + 2*(a_index.x+.5) / ncols,
                  -0.9 + 1.8*(a_index.y+0.5) / nrows);
    // Apply the static subplot transformation + scaling.
    gl_Position = vec4(a*u_scale*position+b, 0.0, 1.0);
    v_color = vec4(a_color, 1.);
    v_index = a_index;
}
"""

FRAG_SHADER = """
#version 120
varying vec4 v_color;
varying vec3 v_index;
void main() {
    gl_FragColor = v_color;
    // Discard the fragments between the signals (emulate glMultiDrawArrays).
    if ((fract(v_index.x) > 0.) || (fract(v_index.y) > 0.))
        discard;
}
"""

One of the changes I made was on the transformation which positions the different points on the window. Now, the plot area starts at -0.9 on the left and stops at 1 on the right; and starts at -0.9 at the top and stops at 0.9 at the bottom, freeing margins.

My question is, how can I add additional elements on the canvas? I would like to add a fix vertical line at e.g. -0.9, I would like to add text element, I would like to add a moving vertical line, an horizontal line, ... I'm guessing each of those elements should have there own VERT and FRAG shaders passed to the python constructor. Is this correct? How do you specify on which Canvas you want to draw, and what would be the different constructor to use for the element cited above?

Mathieu
  • 5,410
  • 6
  • 28
  • 55
  • As a guess, for a moving vertical line, I should create a second program within my class inheriting from Canvas with the corresponding shaders for a moving vertical line. For a fix vertical line and for text; it is not very clear how I should procede. – Mathieu Aug 13 '21 at 14:42
  • You are correct in that you would have to create multiple GL Programs, each with their own shaders, and then in your `on_draw` method you'd have to call each program to be drawn. Drawing text is no small feat. That's why the TextVisual exists. This Visual can be used from a regular Canvas and does not require using the SceneCanvas although I would still recommend the SceneCanvas. As I noted in your other question, you should be able to pass all your line segments to a single LineVisual. What kind of update frequency are you expecting for your signals? Do all 128 update at that frequency? – djhoese Aug 13 '21 at 15:49
  • @djhoese The application is mostly EEG, which can go up to 256 channels at 2 kHz. My plotting window is between 10s and 30s usually; so it would add up to 15M points. A more reasonable scenario is 64 channels at 500 Hz, which adds up to 1M points. I haven't tried yet extremes with 15M points, but with 1M points, the vispy program handles it like a champ. It's super smooth. – Mathieu Aug 13 '21 at 15:57
  • @djhoese I'm not very scared to go towards the low-level, and as long as I know the coordinate system and I have the equation for the position of the elements, I can write the correct program. I don't think I'll go towards the LineVisual; for vertical moving lines, it should be fairly easy. For the text, which I knew was the hardest part, if I can use directly TextVisual withg a regular canvas, as this is what I used, I will give it a try. – Mathieu Aug 13 '21 at 15:59
  • @djhoese For the textVisual, as I would prefer to plot all text string at once (especially the channel names, e.g. 64 channels names..), could you explain what is the text collections mentioned here: https://github.com/vispy/vispy/blob/9ee6f737095d5d4ee6f3d8cc17adb566c6e280d8/vispy/visuals/text/text.py#L281 – Mathieu Aug 13 '21 at 16:03
  • It could be talking about this: https://github.com/vispy/vispy/tree/main/vispy/visuals/collections I'm not worried about you handling low-level gloo/GL stuff. I'm worried about two things: 1. The amount of work you're going to put in and not reusing existing code. Anything in Visuals you can use with your regular Canvas. The Widget classes you can't. 2. Gloo and GL support will be going away in the long run. See the roadmap: https://vispy.org/roadmap.html This probably won't effect you or any other users for a while, but it is something to be concerned about when rolling your own app. – djhoese Aug 13 '21 at 19:25
  • @djhoese Well, crap. The future deprecation of gloo is a real problem for me. This does force me to move to higher-level functions. That's going to be also a lot of work to rework the code around that. I feel like the documentation of Vispy is.. sparse? It's rather difficult to understand what the arguments are doing; even going in the code. I'll give it a shot, but I'll probably end up waiting for future release of Vispy with better documentation. Thanks for all the information, it's a very promising library! – Mathieu Aug 13 '21 at 21:45
  • Do you have examples of things you find difficult to understand? I know we need better documentation, but creating an issue on vispy's github would be a great start to let us know what is difficult to understand. Also about gloo deprecation, this is years and years from now. Most stuff should be easy to port too as we will still be dependent on shaders (just newer versions of GLSL). I mentioned it because I'm not sure the stuff you're doing requires the low-level gloo to work. – djhoese Aug 14 '21 at 11:34
  • @djhoese I will open issues on github with the different documentation difficulties I experienced. I know this feedback and traceability is very important to improve vispy. What I meant by documentation difficulties, is that several time, I have to test what the different arguments, what the different arguments values are doing; as this is not documented. For th deprecation of gloo; I get it that it's in the future, but for my case it would be bweter to be future-proof. – Mathieu Aug 14 '21 at 13:13

0 Answers0