Ascii characters under certain value not being read via serial

Question

I have setup a remote microcontroller connected to my linux PC that spits out a continuous string of characters. I have configured the port using screen (by issuing screen /dev/ttyS0 57600).

I validated the data with od command and got what I expected. Data follows as 8-bit hex values...

    0006040 03 01 09 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006100 00 00 00 00 00 00 00 00 00 00 00 06 0b a5 00 03
    0006120 01 09 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006160 00 00 00 00 00 00 00 00 00 00 06 0b a5 00 03 01
    0006200 09 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006220 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006240 00 00 00 00 00 00 00 00 00 06 0b a5 00 02 01 09
    0006260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006300 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006320 00 00 00 00 00 00 00 00 0f 0c a5 00 02 01 09 00
    0006340 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006360 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006400 00 00 00 00 00 00 00 0f 0c a5 00 02 01 09 00 00
    0006420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006440 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006460 00 00 00 00 00 00 0f 0c a5 00 01 01 09 00 00 00
    0006500 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006520 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006540 00 00 00 00 00 05 0c a5 00 01 01 09 00 00 00 00
    0006560 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006600 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006620 00 00 00 00 05 0c a5 00 01 01 09 00 00 00 00 00
    0006640 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006660 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    0006700 00 00 00 05 0c a5 01 08 01 09 00 00 00 00 00 00

I then create a simple C program to try to read the same data, but all I get from my C program is the following output:

  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 a5:1 
  a5:1

Based on that output, it seems the read function does return that one byte is being read, but every time the value returned is a5 which is the flag byte the micro sends out to the PC to indicate a new stream of fixed-size data.

So my program is right in overwriting the existing "A" value in the read buffer, but it needs to update with the correct data all the time (be it 00, or 01), but not just only a5

What am I doing wrong?

I attached the full code below for someone to use if they want to try to reproduce the problem. and yes, I configured the baud and data bits and stop bits correctly.

  #include <stdio.h>
  #include <stdlib.h>
  #include <fcntl.h>
  #include <time.h>
  #include <unistd.h>

  #define TIMEOUT 5

  int main(){
    char inb[3];  //our byte buffer
    int nread=0;  //number bytes read from port
    int n;        //counter
    int iosz=128; //Lets get 128 bytes
    int fd=open("/dev/ttyS0",O_NOCTTY | O_RDONLY | O_SYNC); //Open port
    for(n=0;n<iosz;n++){
      int s=time(NULL); //Start timer for 5 seconds
      while (time(NULL)-s < TIMEOUT && nread < 1){
        inb[0]='A'; //Fill buffer with bad data
        inb[1]='B';
        inb[2]='C';
        nread=read(fd,inb,1); //Read ONE byte
        if (nread < 0 || time(NULL)-s >= TIMEOUT){
            close(fd); //Exit if read error or timeout
            return -1;
        }
      }
      printf("%x:%d ",inb[0] & 0xFF,nread); //Print byte as we receive it
    }
    close(fd); //program ends so close and exit
    return 0;
  }

Answer 1

Ascii characters under certain value not being read via serial

That's a bogus analysis of your program's flaws.
As it turns out by some coincidence, the largest value received seems to be 0xA5, and the small sample dump seem to always precede that byte with either 0x0B and 0x0C.
If the dump were to show more data, would there eventually be a 0x0D (carriage return) preceding a byte of 0xA5?

So if canonical mode is active (which is typically the default mode) and 0x0D (i.e. a line delimiter character) precedes it, then the 0xA5 would be the first character of a line.
Since your program requests only one character at a time, a canonical read() would only return the first character of the line.

What am I doing wrong?

Your program has two significant bugs.

First, your program neglects to configure the serial terminal, so you don't know whether your program is using canonical or non-canonical mode.

Second, your program is only printing the same results of a single read() repeatedly.
Your analysis that only certain values are being read is inaccurate, because only the result of a single read() is actually printed.
The printf() is outside the scope of the while loop (that performs the reads), so your program does not print what each read() has obtained, but only prints what the first successful read() has obtained.
After that first successful read(), variable nread has value 1, and the comparison of nread < 1 for the while loop will always evaluate to FALSE.
Hence no more read() s will occur after the first successful one, but the results of the first will be repeatedly printed because of the for loop.

So if non-canonical mode is active when you ran your program, and '0xA5` just happens to be the first read(), then that is what would be the printed (many times) by your program.

---

Since the received data is binary and not ASCII text, you need to configure the serial terminal for non-canonical (aka raw) mode.
Then you can persist on inefficiently reading one byte at a time.

Answer 2

I finally figured it out!

After days of brain-wracking, I have found out the ultimate answer.

Yes, I have implemented doing I/O on the serial port with more bytes at once instead of one byte at a time, but that didn't solve it.

What literally solved things were two items:

1. I needed to make this program have high-priority since the serial timing is strict (1 character every 173uS at 56k). For that, I added this at the start of the main function:

   const struct sched_param schedp = {99}; sched_setscheduler(0,SCHED_FIFO,&schedp);

and I also added this in the includes:

#include <sched.h>

This allowed the program to have priority, but surprisingly the thing that topped the cake was memory locking around the sections that needed to be critical, and those critical sections were reading and writing from/to the port.

    mlockall(MCL_CURRENT);  
    //use read() or write() in here with file descriptor as serial port
    munlockall();

and I also added this in the includes:

#include <sys/mman.h>

I considered using MCL_FUTURE as a parameter but that means asking to lock more resources in memory, but after I found that unnecessary, so I will use MCL_CURRENT.

And I don't care if the rest of the processes on my system locks up for 5 seconds, but at least now the data I/O is running 100% better on the serial port.