Live plot using PyQtGraph is drawing even with the device disconnected

Question

I am using Pyserial and PyQtgraph to plot live data. The connection between the device that i am reading the data from (an arduino) and my pc works fine, i mean that i can read the data. The problem comes that when I disconnect the device, data is still drawing in the plot. And, if I let it keep reading, after a while, the plot crashes and i have to start over again.

I was reading some posts and I have found this:

implementing pyqtgraph for live data graphing

So, i think the problem is that in my code, the data is append to a list and then is plot, and that makes it slow and maybe that is why it crashes.

This is my code:

class MyApplication(QtGui.QApplication):
  def __init__(self, *args, **kwargs):
    super(MyApplication, self).__init__(*args, **kwargs)
    self.t = QTime()
    self.t.start()

    self.data = deque()

    self.cnt = 0 

    self.win = pg.GraphicsWindow()

    self.plot = self.win.addPlot(title='Timed data')
    self.curve = self.plot.plot()

    self.tmr = QTimer()
    self.tmr.timeout.connect(self.update)
    self.tmr.start(100)

    self.cnt = 0

    print "Opening port"
    self.raw=serial.Serial("com4",9600)
    print "Port is open"

  def update(self):
    line = self.raw.read()
    ardString = map(ord, line)
    for number in ardString:
        numb = float(number/77.57)
        self.cnt += 1
        x = self.cnt/20
        
        self.data.append({'x': x , 'y': numb}) 
        x = [item['x'] for item in self.data]
        y = [item['y'] for item in self.data]
        self.curve.setData(x=x, y=y)

How can i modify my code to use the code written in the post of above? Or how can I draw the data that is coming without append it to a list?

Sorry, but i am new with PyQtGraph and I am a confused right now. HOpe you can help me.

---------- EDIT ---------

I´ve tried a simpler code like this one:

import serial
import numpy
import matplotlib.pyplot as plt

print "Opening port"
port = "com4"
arduinoData = serial.Serial(port, 9600)

while True:
  if arduinoData.inWaiting()>0:
    print "Reading data"
    arduinoString = arduinoData.read(arduinoData.inWaiting())

    bytes = map(ord, arduinoString)

    for byte in bytes:
        print byte
  else:
    print "There is no data"

So, after it shows the data in the command prompt, I disconnect the device, and I can see that the data are still showing for a few seconds. Then, the "There is no data" text appears. So, what could be the problem? I know that, it is buffered data, but it seems to me that it is the same that is happening with the other code.

---------- EDIT 2 ---------

I finally accomplish to do what i need. Thank you @busfault for all your help and patience.

This is the code for the update method:

def update(self): 
    line = self.raw.read([1])                        
    ardString = map(ord, line)                        
                                                      
    for number in ardString:                          
        numb = float(number/77.57)                    

        self.data.append(numb)                        
        self.yData.append(numb)                       
                                                      
        if len (self.yData)>300 :
            self.yData = []
            self.raw.flush()
        
    self.curve.setData(self.yData)

What i do now is that the data goes to two different lists: self.yData and self.data. In self.yData I can only append up to 300 data items(this is random, I could have chosen 500), and then I flush all the data and "clear" the list to start again.

Whit this I can see live data with no delay and save all of them in another place safe.

Answer 1

I think if you create lists as you go you should see a speed up, if you are set on using the deque, then I would suggest moving the generation of x and y lists outside of the for loop since that is probably where you are spending a LOT of time when it may not be necessary.

def __init__(self, *args, **kwargs):
    super(MyApplication, self).__init__(*args, **kwargs)
    self.t = QTime()
    self.t.start()

    #self.data = deque()
    self.xValues = []
    self.yValues = []


    self.cnt = 0 

    self.win = pg.GraphicsWindow()

    self.plot = self.win.addPlot(title='Timed data')
    self.curve = self.plot.plot()

    self.tmr = QTimer()
    self.tmr.timeout.connect(self.update)
    self.tmr.start(100)

    self.cnt = 0

    print "Opening port"
    ##EDIT CHANGED THIS LINE TO INCLUDE TIMEOUT
    self.raw=serial.Serial("com4",9600, timeout=0)
    print "Port is open"

  def update(self):
    line = self.raw.read()
    ardString = map(ord, line)
    for number in ardString:
        numb = float(number/77.57)
        self.cnt += 1
        x = self.cnt/20

        self.xValues.append(x)
        self.yValues.append(numb)
        #self.data.append({'x': x , 'y': numb}) 
    #x = [item['x'] for item in self.data]
    #y = [item['y'] for item in self.data]
    self.curve.setData(x=x, y=y)

From the PySerial Documentation: https://pythonhosted.org/pyserial/pyserial_api.html#serial.Serial.read

Read size bytes from the serial port. If a timeout is set it may return less characters as requested. With no timeout it will block until the requested number of bytes is read.

and from the Constructor

Possible values for the parameter timeout which controls the behavior of read():

• timeout = None: wait forever / until requested number of bytes are received
• timeout = 0: non-blocking mode, return immediately in any case, returning zero or more, up to the requested number of bytes
• timeout = x: set timeout to x seconds (float allowed) returns immediately when the requested number of bytes are available, otherwise wait until the timeout expires and return all bytes that were received until then.

So by default (timeout=None), when that self.raw.read() gets executed and there is no data it tries to read one byte then waits forever. For a byte to be written.

==================================

I was thinking some more about why your code is crashing after you disconnect. I think I know why, self.tmr keeps generateing signals every 100ms and your slot (update) keeps getting called over and over and with that, self.raw.read() keeps getting called (I think?)

try changing your code in update() as such:

def update(self):
    self.tmr.stop()#Prevent the timer from entering again.
    line = self.raw.read()
    ardString = map(ord, line)
    for number in ardString:
        numb = float(number/77.57)
        self.cnt += 1
        x = self.cnt/20

        self.xValues.append(x)
        self.yValues.append(numb)
        #self.data.append({'x': x , 'y': numb}) 
    #x = [item['x'] for item in self.data]
    #y = [item['y'] for item in self.data]
    self.curve.setData(x=x, y=y)
    self.tmr.start()#restart the timer (resets the timeout)

I don't know if it is critical to keep the 100mSec pulse though? If so you could use a Lock so that when update is called again it doesn't run the same code again. https://docs.python.org/2/library/threading.html#rlock-objects

I think this example shows that it is pretty straight-forward to implement.

import threading

some_rlock = threading.RLock()

with some_rlock:
    print "some_rlock is locked while this executes"

Answer 2

may i suggest the following: upload the data to a db (database).

this would involve however adding more code to your existing program =)

but this is easily achievable using any db: sqlight, couchdb, mongodb..etc etc

or maybe just create a file that tracks the processed values? I noticed that you are not using a list, but rather tuples to store your key:value pairs within it.

    self.data.append({'x': x , 'y': numb}) 
    x = [item['x'] for item in self.data]
    y = [item['y'] for item in self.data]
    self.curve.setData(x=x, y=y)

So for option number two

tracking_file = open("filename.txt", "w") #w indicates write
tracking_file.writelines(the data) #instead of appending to a tuple or list
track_file.close()

this sequence opens a file and writes data to it, for more info regarding input/output https://docs.python.org/2/tutorial/inputoutput.html

and subsequently, you can read the data from files you created, the files will not be deleted if your program crashes, and you will be able to resume from where your program left before crashing or disconnecting by opening the file in read mode, to check the last input and continue adding values...