1

I've got a hardware client1 who's line of data acquisition cards I've written a Linux PCI kernel driver for.

The card can only communicate 1-4 bytes at a time depending on how the user specifies to utilize it, given this, I utilize ioctl for some of the functionality, but also make use of the file_operations structure to treat the card as a basic character device to give the user of the card the ability to just use read or write if they just want simple 1-byte communication with the card.

After discussing the driver with the client, one of their developers is in the understanding that treating the card as a character device by using open/read/write will introduce latency on the PCI bus, versus using open/ioctl.

Given that the driver makes no distinction how it's opened and the ioctl and read/write functions call the same code, is there any validity to this concern?

If so, how would I test the bus latency from my driver code? Are there kernel functions I can call to test this?

Lastly, wouldn't my test of the bus only be valid for my specific setup (kernel settings, platform, memory timing, CPU, etc.)?

1: they only have 2 other developers, neither of which have ever used Linux

txtechhelp
  • 6,625
  • 1
  • 30
  • 39
  • Your wording confuses me. I could imagine that some forms or patterns of access are *more affected* by PCIe bus latency than others, but I don't see how they could *introduce* bus latency. Isn't the latency you're talking about a hardware characteristic? – John Bollinger Dec 18 '18 at 19:23
  • @JohnBollinger .. that's basically my question. My code, as is, wouldn't _introduce_ bus latency because the `ioctl` and `read/write` are the same code, so any bus latency is due to the card or PCI bus itself, not the driver (or necessarily even the kernel). I've tried to explain this to the client's hardware guys and the developer who thinks this, but the developer wants me to give proof, and I'm just the contractor .. so I'm trying to find out how I can test the timing of the bus through my driver based on the function I utilize, or provide other proofs to this fact. – txtechhelp Dec 18 '18 at 19:31
  • The part that really made my ears perk up was the contrast between "communicates 1-4 bytes at a time" and "1-byte communication". If the `read` and `write` implementations indeed operate only a single byte at a time, but the hardware supports four at a time, then they will indeed suffer from more latency than is necessary. Whether you provide an `ioctl` that could do better is unclear. – John Bollinger Dec 18 '18 at 19:33
  • 1
    Frankly, it's also a bit surprising to me that your driver exposes ioctls that can serve the same purpose as `read` and `write`. Of course, the `ioctl` interface is very generic and can support that, but it normally is used for providing access to device parameters, not for performing I/O. – John Bollinger Dec 18 '18 at 19:39
  • @JohnBollinger .. I edited my question to make that a little more clear; the card _can_ communicate 1, 2 or 4 bytes at a time, depending on how the user specifies, so a 1-byte `ioctl` call is the same is a 1-byte `read` (at least that's the question) ... as an aside, the `ioctl` to communicate is something they wanted in the driver :/ – txtechhelp Dec 18 '18 at 19:40
  • What is the data transfer rate of the device? – Craig Estey Dec 18 '18 at 19:42
  • It may be worth having a talk with the concerned developer. Reading between the lines here, I'm guessing that your client's devs have been involved in preparing Windows drivers for the hardware, and that you have been engaged to prepare a Linux driver. The client's devs might then have some specific experience with the hardware that you could benefit from, with their concerns springing from that. It doesn't sound like your `ioctl` would exhibit different performance characteristics than your `read` and `write`, but perhaps they could *all* be better. – John Bollinger Dec 18 '18 at 19:48
  • @CraigEstey they have roughly 50 different PCI DAQ cards, so it can vary, but those rates are card specific and documented in their specs, but they all read/write via the same address', so the driver doesn't need to distinguish when it's a read/write operation. Although, that _might_ be where this dev's concern come from. – txtechhelp Dec 18 '18 at 19:49
  • The transfer rate matters. For a few thousand bytes/second it doesn't matter. For millions, using `ioctl` may be necessary, because you'd need to set up zero-copy buffers mappable in userspace, kernel, and device space. At one company, I did `ioctl` and another developer did `read/write` for a DMA video device. We had to switch it to `ioctl` when the `read/write` [being a stream of sorts] couldn't adequately synchronize. Also, you'll need `ioctl` to be able to set the BAR registers of the device to get the correct mode for maximum transfer speed – Craig Estey Dec 18 '18 at 20:04

1 Answers1

2

I suspect the client's developer is slightly confused. He's thinking that the distinction between using read or write versus ioctl corresponds to the type of operation performed on the bus. If you explain to him that this is just a software API difference and either option performs the exact same operation on the bus, that should satisfy them.

David Schwartz
  • 179,497
  • 17
  • 214
  • 278
  • Based on the other comments and this answer, I'm going to try and reword my explanation to the hardware team, and then have an offline conversation with the dev to ascertain where the concern is, and maybe even walk them through it all ... They've seen the code, so I'm hoping it's just a matter of a deeper conversation .. thanks! – txtechhelp Dec 18 '18 at 19:53
  • 1
    It's an easy thing to misunderstand, particularly for someone who works on hardware a lot and doesn't work on drivers much. It's easy to think that because you call a function named `read`, that means a read operation is performed on the bus. And so on. – David Schwartz Dec 18 '18 at 19:54
  • _that means a read operation is performed on the bus_ .. great insight! I'm guessing that's where the confusion is likely coming from. – txtechhelp Dec 18 '18 at 19:56