Serial port data received handled too slowly

Question

I am reading data from an arduino at a baud rate of 115200. The data comes in as a string on its own line in the format: <ID,Name/Data>.

I believe that the problem with my code is that it is not handling the incoming data fast enough and the incoming data is being forced to wait for old data to be processed.

The incoming string is split into the three separate categories (ID, Name, Data) and added to a data table called dtFromGrid which is bound to dataGridView1.

Is there any errors or suggestions for how to improve my code performance? Would a separate thread for the handling function work better than BeginInvoke?

serialPort1.DataReceived += new SerialDataReceivedEventHandler(serialPort1_DataReceived);

    private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
    {
        string inData = serialPort1.ReadLine();
        if (PauseButton.Text == "Pause" && inData.StartsWith("<"))
        {
            try
            {
                this.BeginInvoke(new SetGridDeleg(DoUpdate), new object[] {inData});
            }
            catch
            {
            }
        }
    }

    private void DoUpdate(string inData)   //inData passed in so that Serial port read only once
    {
        if (dtFromGrid == null)
        {
            dtFromGrid = new DataTable();
            dtFromGrid.Columns.Add("Time", typeof(String));
            dtFromGrid.Columns.Add("ID", typeof(String));
            dtFromGrid.Columns.Add("Name", typeof(String));
            dtFromGrid.Columns.Add("Data", typeof(String));
        }

        DataRow dr = dtFromGrid.NewRow();
        TimeSpan ts = stopWatch.Elapsed;
        dr["Time"] = String.Format("{0:00}:{1:00}:{2:00}.{3:000}",
        ts.Hours, ts.Minutes, ts.Seconds,
        ts.Milliseconds);
        dr["ID"] = inData.Split(new char[] { '<', ',' })[1];
        dr["Name"] = inData.Split(new char[] { ',', '/' })[1];
        dr["Data"] = inData.Split(new char[] { '/', '>' })[1];
        dtFromGrid.Rows.InsertAt(dr, 0);

        //Replace old data with new data if ID's are the same to showo list of only newest data per each ID
        if (NewestButton.Text == "Chronological")
        {
            for (int i = 1; i < dataGridView1.Rows.Count; i++)
            {
                if (dtFromGrid.Rows[i].ItemArray[1].ToString() == dtFromGrid.Rows[0].ItemArray[1].ToString())
                {
                    dtFromGrid.Rows[i].Delete();
                    break;
                }
            }
        }

        //Keep a maximum of 50 rows of data
        if (dtFromGrid.Rows.Count == 51)
        {
            dtFromGrid.Rows[50].Delete();
        }
        dtFromGrid.AcceptChanges();
        dataGridView1.DataSource = dtFromGrid;

        //keep focus of dataGridView on top row
        dataGridView1.CurrentCell = dataGridView1.Rows[0].Cells[0];

        // add newest row to a logfile if the user has set one
        if (logFile != "")
        {
            using (StreamWriter sw = File.AppendText(logFile))
            {
                DataRow row = dtFromGrid.Rows[0];
                object[] array = row.ItemArray;
                int col = 0;
                for (col = 0; col < array.Length - 1; col++)
                {
                    sw.Write(array[col].ToString() + "\t|\t");
                }
                sw.Write(array[col].ToString());
                sw.WriteLine();
                sw.Close();
            }
        }
    }

Update

I am now using a separate thread as suggested but I am having errors with invoking inside of that thread. I get multiple errors at random but the most common is "Index out of range." My invoke code is as follows: this.Invoke((MethodInvoker) delegate { dtFromGrid.AcceptChanges(); dataGridView1.DataSource = dtFromGrid; dataGridView1.CurrentCell = dataGridView1.Rows[0].Cells[0]; });

Answer 1

You store the data in a queue and offload the work to a secondary thread. This only works if, on the average, you are able to process the data at the rate it is coming in. Otherwise, the size of the queue will keep growing as you fall behind.

First, start with a wrapper around Queue<T> that will allow one thread to write to the queue and another to read from it in a thread safe manner. Also, allows the reader thread to block waiting for data.

public class ThreadedQueue<T>
{
    private readonly Queue<T> _queue = new Queue<T>();
    private readonly ManualResetEvent _notEmptyEvt = new ManualResetEvent(false);

    public WaitHandle WaitHandle { get { return _notEmptyEvt; } }

    public void Enqueue(T obj)
    {
        lock (_queue)
        {
            _queue.Enqueue(obj);
            _notEmptyEvt.Set();
        }
    }

    public T Dequeue()
    {
        _notEmptyEvt.WaitOne(Timeout.Infinite);
        lock (_queue)
        {
            var result = _queue.Dequeue();
            if (_queue.Count == 0)
                _notEmptyEvt.Reset();
            return result;
        }
    }
}

In your serial port handler, write the data into the queue:

private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
    string inData = serialPort1.ReadLine();
    if (PauseButton.Text == "Pause" && inData.StartsWith("<"))
    {
        _queue.Enqueue(inData);
    }
}

In the secondary thread, read from the queue and do the invoke to the GUI thread:

private void ThreadProc()
{
    while (true)
    {
        string inData = _queue.Dequeue();
        this.Invoke(new SetGridDeleg(DoUpdate), new object[] {inData});
    }
}

Start up the secondary thread like this:

Thread th = new Thread(ThreadProc);
th.IsBackground = true;
th.Start();

Of course you'll need to create an instance of the queue:

ThreadedQueue<string> _queue = new ThreadedQueue<string>();

Answer 2

I normally design a SerialService class to manage the SerialPort. Below is a simple version of the SerialService class.

The role of the SerialService class is to read the serial buffer as fast as possible. This clears the buffer and prevents any serial port errors. This raw data is then passed to the parser.

The trick for performance is in your parser. YourParser should also be fast in formatting the raw data into the string that you are expecting. Once your data is parse you may use a callback or an event. With a callback or event your parser will continue to parse new arriving data. YourParse is now a testable class.

Once you have your good data from the parser's callback use BeginInvoke to send the data to the main thread where your ui can then display it.

If you are not in the main UI thread and you try to update the UI from another thread you will have the cross theading problem.

Good luck.

class Program
{
    private static YourDataParser _parser;
    static void Main(string[] args)
    {
        _parser = new YourDataParser();
        var serial = new SerialService("COM1");
        serial.DataReceived += serial_DataReceived;
    }

    static void serial_DataReceived(object sender, DataReceivedEventArgs e)
    {
        _parser.HandleTheData(e.Data, good =>
        {
            // here is your good data
            // This is not the main thread invoke your UI from here with the good data
            // Use BeginInvoke to invoke the main thread
        });
    }
}

public class YourDataParser
{
    private List<byte> _buffer = new List<byte>(); 

    public void HandleTheData(byte[] rawdata, Action<string> goodData)
    {
        _buffer.AddRange(rawdata);

        foreach (var b in _buffer)
        {
            var thechar = (char) b;

            // handle your raw data... like look for the character '<'
            // or look for the end of line this would be CR (0x0D) LF (0x0A) 
            // you can reference the ASCII table for the characters byte values
        }

        // and return the good data 
        var data = "your good data after parsing it";
        goodData(data);
    }
}

public class DataReceivedEventArgs : EventArgs
{
    public DataReceivedEventArgs(byte[] data)
    {
        Data = data;
    }

    public byte[] Data { get; private set; }
}

class SerialService
{
    public event EventHandler<DataReceivedEventArgs> DataReceived;

    private SerialPort _port;
    public SerialService(string comm)
    {
        _port = new SerialPort(comm)
        {
            // YOUR OTHER SETTINGS HERE...
            ReceivedBytesThreshold = 1 // I think is better to increase this number if you know the minimum number of bytes that will arrive at the serial port's buffer
        };

        // Note that the ReceivedBytesThreshold is set to 1. 
        // That means that the port_DataReceived event will fire with a minimun of 1 byte in the serial buffer
        _port.DataReceived += port_DataReceived;
    }

    void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
    {
        if (e.EventType != SerialData.Chars) return;

        while (_port.IsOpen & _port.BytesToRead != 0)
        {
            // important to get all the bytes off the buffer
            var size = _port.BytesToRead; 
            var buffer = new byte[size];

            var sizeRead = _port.Read(buffer, 0, size);

            OnDataReceived(buffer);
        }

    }

    protected virtual void OnDataReceived(byte[] data)
    {
        var ev = DataReceived;
        if (ev != null) ev(this, new DataReceivedEventArgs(data));
    }
}

Answer 3

As you said your code is slowing the data reception. you can solve your problem by queuing your data to a queue list and a background process will be processing this list one by one. another approach is to create a new thread on the reception of each data batch.
Example (second approach)

private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
    {
       string inData = serialPort1.ReadLine();
       System.Threading.Thread T = new System.Threading.Thread(new System.Threading.ParameterizedThreadStart(ProcessData));
       T.Start(inData);
}
public void ProcessData(Object data)
{
....
}