How can I extract my required information from a continuous stream of data.?

Question

I'm working on a project in which a continuous stream of data is coming on the serial port through Arduino in which I have to extract my required information. I have include "AA" in the beginning of my required data and "55" at the ending.

Arduino Serial Monitor output -

AA055
AA155
AA255
AA355
AA455

Arduino code-

int i=0;
void setup()
{  
 Serial.begin(9600);    // Open serial connection at a baud rate of 9600
}

void loop()
{ 
 Serial.print("AA");
 Serial.print(i);
 Serial.println("55");

 i++;
 delay(100);
 }

PYTHON CODE

import serial
import time
import csv

baudrate = 9600
port = 'COM2'  # set the correct port before run it

serial = serial.Serial(port=port, baudrate=baudrate)
serial.flushInput()
with open("test_data.csv","a") as f:
    writer = csv.writer(f,delimiter=",")
    writer.writerow(["TIME STAMP", "DATA"])

while True:
    try:
        ser_bytes = serial.readline()
        decoded_bytes = float(ser_bytes[0:len(ser_bytes)-2].decode("utf-8"))
        print(decoded_bytes)
        aq = time.strftime('%a %H:%M:%S')
        print(aq)
        with open("test_data.csv","a") as f:
        writer = csv.writer(f,delimiter=",")
        writer.writerow([time.strftime('%a %H:%M:%S'),decoded_bytes])
    except:
        print("Keyboard Interrupt")
        break

I want the python code to extract my information by detecting the "AA" at the beginning and "55" at the end.

As far as I can understand, `decoded_output` is the output from the serial output. Right? — MaJoR, Dec 27 '18 at 07:58
Yes, but since a continuous stream of data is coming from arduino I want to extract only my required information — Ansari Aquib, Dec 27 '18 at 08:00
Then you would have to just run some operations on `decoded_output` in the while loop to extract the information. I will post an answer on this. — MaJoR, Dec 27 '18 at 08:03
It would be of great help, because I'm just a newbie in python. — Ansari Aquib, Dec 27 '18 at 08:12

MaJoR · Accepted Answer · 2018-12-27T08:18:34.850

0

Assuming you need to extract the information from the serial output, which starts with "AA" and ends with "55", you can make some modifications to your python code like this:

...
print(decoded_bytes)

#First check whether the length of the serial output is at least 4. 
if len(decoded_bytes) >= 4:
    if decoded_bytes[0:2] == "AA" and decoded_bytes[len(decoded_bytes)-2:len(decoded_bytes)] == "55":
        print(decoded_bytes) # The information from serial output, between "AA" and "55"

aq = time.strftime('%a %H:%M:%S')
...

Suppose the serial output is like:

AA155
AA255
AB155
AA235

This piece of code will only print the outputs which start with "AA" and end with "55". That is the first two cases.

We are checking whether the length is greater than 4 because the information will be contained within "AA" and "55".

Important thing to note here is the string slicing operator. You can read it up here. It basically returns a part of the string, within some indices.

edited Dec 27 '18 at 08:18

answered Dec 27 '18 at 08:13

MaJoR

954
7
20

I'm not getting the output after writing the piece of code you provided. Can you provide the processing using while loop ... I know how to do it in C, but im not getting how to do it in python – Ansari Aquib Dec 27 '18 at 09:13
if decoded_bytes[0:2] =="AA": ................after this line the code is not working ,if i removed it im getting the normal output i.e. AA055, AA155,etc – Ansari Aquib Dec 27 '18 at 09:34
decoded_bytes = ascii(ser_bytes[0:len(ser_bytes)-2].decode("utf-8")) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 2: invalid start byte – Ansari Aquib Dec 27 '18 at 09:38
You are getting an error because you are not reading the serial data correctly. The error is in `decoded_bytes = float(ser_bytes[0:len(ser_bytes)-2].decode("utf-8"))` line. Refer to [this](https://stackoverflow.com/questions/35936644/python-read-from-the-serial-port-data-line-by-line-into-a-list-when-available) question on how to read the serial data in Python correctly. – MaJoR Dec 27 '18 at 09:42

How can I extract my required information from a continuous stream of data.?

1 Answers1