8

I'm sorry if this is a very vague question, but I can't seem to formulate it properly to find anyone else with this issue. The main question is, once you have a serial connection established between two devices, how do you use that connection to implement two way communication?

An example might help. Suppose you have a temperature sensor as an embedded device using a microcontroller and firmware written in C. You have a serial port connection from that sensor to a computer and some software on the computer to interface with it, say a C++ application. I understand how to setup the serial ports on both sides and read and write single bytes of data between the two devices. The real question is what convention do you use to communicate between the two devices?

Assume your requirements are as follows:

1.) You need to be able to send a command to take a single temperature reading from the embedded device and send it to the computer for display purposes.

2.) You need to send commands to make the sensor start and stop streaming temperature values.

3.) You need a set of commands to set various aspects in the firmware such as streaming rate, stream on startup, blink leds, etc.

4.) You need some sort of structure to send complicated forms of data to the computer, perhaps an array of battery voltage readings.

Ways to Accomplish This

It seems there are a couple ways people tend to do this:

Simple String API:

The most common, from dealing with third party sensors, seems use a simple string based API, such that the commands for starting and stopping streams might be "SS,1\r" and "SS,0\r" respectively. In this case you would have to read from the serial port until you get the "\r" character and then parse the data you got to see if it has a command (left of comma) and parameters (right of comma). This works for scenarios 1 to 3 above but doesn't make scenario 4 very easy.

JSON String API:

This works the same as the one above but instead of passing your parameters as simple values, you pass JSON objects that can represent complicated structures of data. Hence you could send the array of battery voltages as a JSON array. This method seems to cover all use cases 1-4 above. But JSON sends strings and it is harder to parse using embedded c. It would work wonders for the computer side which could use a higher level language such as Java that has libraries for reading JSON data.

Packet Style API:

This is the solution we accepted that I am now kind of regretting. It involves sending a structured packet convention of bytes for each piece of data we send. The packet structure is shown below.

[0xFF][0xFF][ID][CMD][D0][D1][D2][D3][D4][D5][D6][D7][0xEE][0xEE][0xEE]

With this structure we send a header and footer (0xFF's and 0xEE's) for validating packet completeness, an id for sending sequential packets (for transmitting an array of data), a data array which we can use to pack longs, floats, ints, etc, and a command byte (CMD) which can be used by the device to determine how to parse the data payload (D0-D7).

So I ask, what is the most preferred way to communicate over a serial port? Is there any other ways I am missing? I have been doing a lot of web development lately and it seems JSON is a nice abstract transmission system, but has its limitations because you have to do a lot more string parsing, which is a bit complicated on the firmware side.

Kara
  • 6,115
  • 16
  • 50
  • 57
JBausmer
  • 914
  • 9
  • 11
  • What's wrong with the approach you used, other than it being proprietary? What do you mean by "preferred?" There are small, reliable parsers for JSON that you can use on an embedded device: http://zserge.bitbucket.org/jsmn.html – Robert Harvey Sep 05 '13 at 19:30
  • Most JSON is used over a network where the text of the JSON message is wrapped in packet(s) and checksummed and verified to make sure it gets to where you are sending it. Raw serial bytes are not guaranteed to transmit correctly. (Usually failures are 1 or more bits in sequence. Maybe some whole bytes.) You need to do something to verify the integrity of the message and to verify that what you send actually got there. What you use should do this. – Lee Meador Sep 05 '13 at 19:38
  • For one, it limits use to 256 command types. To get around this we broke our convention of sending data and commands in single packets. We now send a packet who's data sets the parameter ID we want to set, and a separate packet that sends the value to set that parameter to. It has in a sense implemented state into our delivery system, the firmware has to remember an ID in ram until it gets the data to set that parameter to. We use 2 packets where we used to use 1, make failure more likely. By preferred I mean surely someone else has a better way to send and parse structured data. – JBausmer Sep 05 '13 at 19:45
  • 1
    what about json-rpc that deals with both send and receive-ack? and it is also a common method. – Shawn Shaw Jul 07 '17 at 04:24

2 Answers2

5

The biggest problem here is, that there is a lack of standards. Almost everyone implements their own protocol for constraint devices or when it comes to low level transports like UART / Serial Line. But with a growing IoT trend this hopefully starts to change.

Please have a look at SLIP and SLIPMUX to send packet oriented protocols over serial line.

No matter if you send JSON, XML or anything else, you most likely need some stop flag to terminate your messages. Often \n is used for sake of readability in combination with ASCII encoded content, but that's not the best for binary encoded machine 2 machine communication.

SlipMux is backwards compatible to SLIP (for IPv4 and IPv6 packets) but also supports new message types like CoAP and Diagnostic (human readable) messages. You can simply implement your own packet types like "JSON" or "XML" messages.

As a protocol I can recommend CoAP in combination with SlipMux. CoAP is very similar to HTTP but much more lightweight and thus easy to deal with for most developers.

Community
  • 1
  • 1
Tarion
  • 16,283
  • 13
  • 71
  • 107
3

The binary packet format is attractive because of its apparent simplicity and efficiency. Furthermore, some coomunication links are already sending data as packets of fixed size (USB, TCP/IP, file systems, etc.) but, if I had to do it differently, I would not go that way again because of the following drawbacks:

  • not human readable (unless you want to obfuscate things and protect your protocol :-) )
  • computer architecture dependant: think about endianness problems
  • toolchain dependancies: ex., how compiler X packs data structures versus compiler Y?
  • link dependencies: if the link changes, what if there is an odd fit between protocol packet size and the new link packet size?

JSON would be my way now: ASCII data transmissions are less efficient than binary transmissions, but its convenience and portability offset that, unless the application has severe bandwidth limitation. I personnally wrote a JSON parser for Arduino Uno board and it works in less than 2 kb of data RAM, Including system stack. OK, it may choke on deeply nested JSON transmissions but it was good enough to strip key elements from a Twitter tweet packet, and proved that it could work on tiny systems.

Yves McDonald

user2918341
  • 31
  • 2