3

I need to transfer data from a bare metal microcontroller system to a linux PC with 2 MBaud. The linux PC is currently running a 32 bit Kubuntu 14.04.

To archive this, I'd tried to use a FT232R based USB-UART adapter, but I sometimes observed lost data.

As long as the linux PC is mainly idle, it seems to work most time; however, I see rare data loss.

But when I force cpu load (e.g. rebuild my project), the data loss increases significantly.

After some research I read here, that the FT232R consist of a receive buffer with a capacity of only 384 Byte. This means, that the FT232R has to be read out (USB-polled) after at least every 1,9 ms. Well, FTDI recommends to use flow control, but because of the used microcontroller system, I'm fixed to cannot use any flow control.

I can live with the fact, that there is no absolutely guarantee for having no data loss. But the observed amount of data loss is quiet too heavy for my needs.

So I tried to find a way to increase the priority of the "FT232 driver" on my linux, but cannot find how to do this. It's not described in the AN220 FTDI Drivers Installation Guide for Linux and the document AN107 FTDI Advanced Driver Options has a capter about "Changing the Driver Priority" but only for windows.

So, does anybody know how to increase the FT232R driver priority in linux?

Any other ideas to solve this problem?

BTW: As I read the FT232H datasheet, it seems that this comes with 1 KiB RX buffer. I'd order one just now and check out its behaviour. Edit: No significant improvement.

Joe
  • 3,090
  • 6
  • 37
  • 55
  • You can look at `drivers/usb/serial/ftdi_sio.c`, and contact author of this driver about your issue – fghj Nov 09 '15 at 03:42
  • Joe, Just a quick thought, have you considered wrapping your UART connection with some scaled down version of a TCP protocol? – Borgboy Nov 09 '15 at 04:35
  • @Borgboy: Did you mean, I should use a Tcp-Uart module instead of Usb-Uart? Could you recommend one which is capable for 2 MBaud ? – Joe Nov 09 '15 at 16:57
  • @Joe: No, I mean you might want to consider a software based handshake mechanism between the Usb-UART and your microcontroller. I do think increasing the RX buffer will help considerably. Out of curiosity, does your microcontroller have external RAM? Here's a EvK that I have that provides the MCU, SRAM, and UART2USB all on one board: [link](http://www.mouser.com/ds/2/368/C8051F064-EK-514402.pdf). It's designed specifically for high data collection and transfer. – Borgboy Nov 10 '15 at 00:38
  • Ok, understand. But unfortunately, I have absolutely no control over the data which will be write out by the µC. The µC just pumps it's data to uart TX pin and I have to (try to) catch it all. – Joe Nov 11 '15 at 08:51
  • 1
    what about implemeting RTS / CTS flow control? – Luka Rahne Nov 11 '15 at 10:53
  • Of course, this is normally the preferred way. But in this case, I'm not allowed to stop the µC's output flow. – Joe Nov 11 '15 at 16:06
  • "I'm not allowed to stop the µC's output flow." Who is not allowing that? It makes zero sense. You can't implement reliable communications without flow control, period. You are literally sending data into a full buffer on the FTDI end of things. This will corrupt your data unless you use a framed communication protocol with error checking and recovery (or at least resilience). **Every FTDI implementation that doesn't at least route the RTS# pin to the MCU is fundamentally broken in a way that can't be fixed.** – Kuba hasn't forgotten Monica Nov 13 '15 at 17:07
  • The bare metal device sends debug / logging output over this uart and this is not under my control. For now I tend to put an additional µC between this device and the FT232R with a large enough buffers (e.g. 64k) and then I can use rts/cts between this one and the FT323R. – Joe Nov 14 '15 at 22:34
  • The FT232R is limited to 0.9 Mbaud on Windows. The CP2104 is supposed to do 1.8 MBaud. – MadeInTheUSB Feb 06 '17 at 00:59
  • I'm running FT232R at up to 3 Mbaud on Linux / FreeBSD / macOS and FT232H at up to 12 Mbaud. – Joe Feb 07 '17 at 01:10

1 Answers1

3

If you want reliable data transfer, there is absolutely no way to use any USB-to-serial bridge correctly without hardware flow control, and without dedicating at least all remaining RAM in your microcontroller as the serial buffer (or at least until you can store ~1s worth of data).

I've been using FTDI devices since FT232AM was a hot new thing, and here's how I implement them:

  1. (At least) four lines go between the bridge and the MCU: RXD, TXD, RTS#, CTS#.

  2. Flow control is enabled on the PC side of things.

  3. Flow control is enabled on the MCU side of things.

  4. MCU code is only sending communications when it can fit a complete reply packet into the buffer. Otherwise, it lets the PC side of it time out and retry the request. For requests that stream data back, the entire frame is dropped if it can't fit in the transmit buffer at the time the frame is ready.

  5. If you wish the PC to be reliably notified of new data, say every number of complete samples/frames, you must use event characters to flush the FTDI buffers to the hist, and encode your data. HDLC works great for that purpose and is documented in free standards (RFCs and ITU X and Q series - all free!).

  6. The VCP driver, or the D2XX port bring-up is set up to have transfer sizes and latencies set for the needs of the application.

  7. The communication protocol is framed, with CRCs. I usually use a cut-down version if X.25/Q.921/HDLC, limited to SNRM(E) mode for simple "dumb" command-and-respond devices, and SABM(E) for devices that stream data.

The size of FTDI buffers is immaterial, your MCU should have at least an order of magnitude more storage available to buffer things.

If you're running hard real-time code, such as signal processing, make sure that you account for the overhead of lots of transmit interrupts running "back-to-back". Once the FTDI device purges its buffers after a USB transfer, and indicates that it's ready to receive more data from your MCU, your code can potentially transmit a full FTDI buffer's worth of data at once.

If you're close to running out of cycles in your realtime code, you can use a timer as a source of transmit interrupts instead of the UART interrupt. You can then set the timer rate much lower than the UART speed. This allows you to pace the transmission slower without lowering the baudrate. If you're running in setup/preoperational mode or with lower real-time task load, you can then trivially raise the transmit rate without changing the baudrate. You can use a similar trick to pace the receives by flipping the RTS# output on the MCU under timer control. Of course this isn't a problem is you use DMA or a sufficiently fast MCU.

If you're out of timers, note that many other peripherals can also be repurposed as a source of timer interrupts.

This advice applies no matter what is the USB host.

Sidebar: Admittedly, Linux USB serial driver "architecture" is in the state of suspended animation as far as I can tell, so getting sensible results there may require a lot of work. It's not a matter of a simple kernel thread priority change, I'm afraid. Part of the reason is that funding for a lot of Linux work focuses on server/enterprise applications, and there the USB performance is a matter of secondary interest at best. It works well enough for USB storage, but USB serial is a mess nobody really cares enough to overhaul, and overhaul it needs. Just look at the amount of copy-pasta in that department...

Kuba hasn't forgotten Monica
  • 95,931
  • 16
  • 151
  • 313