Character conversion errors when using Pyserial

Question

I've been writing a script that pulls data from a serial device at regular intervals. I've got everything working, except that the responses I receive from the device contain many question marks. Since the formatting is correct on what I'm getting according to documentation for the device (correct number of characters, placement of commas etc.) I'm pretty sure that my code is interpreting the characters wrong somehow. Here is my code:

import serial, time

ser = serial.Serial(
    '/dev/cu.usbserial',
    baudrate = 9600, 
    bytesize = 8, 
    timeout = 3, 
    stopbits = serial.STOPBITS_ONE, 
    parity = serial.PARITY_ODD, 
)

if(ser.isOpen() == False):
    ser.open()

print 'Enter your commands below.\r\nInsert "exit" to leave the application.'


while 1 :
    # get keyboard input
    input = raw_input(">> ")

    if input == 'exit':
        ser.close()
        exit()
    else:
        ser.write(input + chr(13) + chr(10))
        out = ''

        time.sleep(3)

        while ser.inWaiting() > 0:
            out += ser.read(1)

        if out != '':
            print '>>' + out

and sample output from the user manual:

ENTER COMMAND? KRDG? 
RESPONSE: +273.15

ENTER COMMAND? *IDN?
RESPONSE: LSCI,MODEL331S,123456,020399

and what the output looks like:

COMPUTERNAME$ python serialTest.py
Enter your commands below.
Insert "exit" to leave the application.
>> KRDG?
?>??4?2?8
>> KRDG?   
?>??4?2??
>> *IDN?
?>L?CI,?O?EL??1?,??????,12?4?7

Of specific note is that fact that on the last line in my output, the '>>' from my code which is concatenated with the output read from the serial device is changed to '?>', which is a little perplexing. For reference, the manual specifies that ports should be configured as follows:

*Baud Rate: 9600 *Character Bits: 1 Start, 7 Data, 1 Parity, 1 Stop *Parity: Odd *Terminators: CR(0DH) LF(0AH)

I've tried every reasonable permutation of port settings. It looks like it might be a parity issue (as I've seen other with similar looking output which is attributed to parity errors online), but I've tried all available parity settings. I'm wondering if it might be something to do with the start bit, since this is the only thing I'm unable to control via Pyserial.

Sorry for the long winded post, and thanks in advance for any/all help!

Konstantin Purtov · Accepted Answer · 2016-03-10T13:58:07.193

1

The code looks correct. try to use ord to understand which bytes you get(Byte may not correspond to ASCII symbol), like this:

out = []
while self.conn.inWaiting() > 0:
    out.append(ord(self.conn.read(1)))
if len(out) > 0:
    print out

Edit:

the answer is (for 7bit ASCII) :

out.append(self.conn.read(1) & 0x7F)

edited Mar 10 '16 at 13:58

answered Mar 01 '16 at 20:32

Konstantin Purtov

799
1
11
19

i will try this next time i'm in the lab. i'm assuming your above code is a general example of ord's function and that i would switch my code to display the result of ord for each character? – oirectine Mar 02 '16 at 21:25
Yes. You should switch only one row 'out.append(ord(self.conn.read(1)))'. You will see the characters as HEX numbers(bytes as [HEX](https://simple.wikipedia.org/wiki/Hexadecimal_numeral_system)) and compare that with device documentation and with character encoding. – Konstantin Purtov Mar 03 '16 at 11:03
So checking the output with ord() gives the following: "[76, 211, 67, 73, 44, 205, 79, 196, 69, 76, 179, 179, 49, 211, 44, 179, 179, 211, 193, 179, 181, 44, 49, 50, 176, 52, 176, 55, 13, 138]". Definitely showing values outside the ASCII range, but not really sure where to go from there; it should be sending ASCII according to documentation. The strange thing is the characters which do "make it through" are correct. – oirectine Mar 07 '16 at 22:43
It is the string (LSCI,MODEL331S,33SA35,120407). I think it is your device=). Try to use the 7-bit ascii format. Like this 'out.append(self.conn.read(1) & 0x7F)' – Konstantin Purtov Mar 09 '16 at 10:07
THANK YOU! if you don't mind me asking, the problem here was that python was interpreting the characters wrong? does this not correspond to telling pyserial to use 7 bit encoding? just trying to understand where the problem was for future projects. thanks again. – oirectine Mar 09 '16 at 19:45
You are welcome! I think it is not the pyserial problem. Maybe connection settings is wrong or the device use the last bit for additional information, or device use the other version ascii encoding. If my advice has helped you, mark it as a solution, please, or ask the next question=). Also you can see this link : https://m.reddit.com/r/learnpython/comments/46zhyh/problems_with_character_conversion_when_using/ – Konstantin Purtov Mar 10 '16 at 14:19

score 0 · Answer 2 · answered Mar 02 '16 at 10:17

0

Your code is using 8 bits databytes, try changing it to 7 as the documentation says:

bytesize = serial.SEVENBITS,

What hardware serial interface are you using? Remember some need the ground to be connected between the the two to work properly.

answered Mar 02 '16 at 10:17

J. P. Petersen

4,871
4
33
33

a previous version of the code had seven bits and the same issue existed unfortunately. it's connect using an rj45 cable and two RJ45-DB9 adaptors, wired for null-modem with ground wire. – oirectine Mar 02 '16 at 21:23
Maybe the baud rates (clocks) on the two sides differ a slightly, this could result wrongly received or missed bits, hence the question marks. The start bit is mandatory, and is used to align the clock signals. Havn't heard about different lengths of start bits. The serial signal is asynchronous, there is a clock signal in each end, which needs to be aligned by the start bit. Is the RJ45 long? – J. P. Petersen Mar 03 '16 at 12:43
I believe it's 50 ft, which is their maximum recommended length. I will try with a shorter cable as well. – oirectine Mar 03 '16 at 19:06

Character conversion errors when using Pyserial

2 Answers2