Win32 IOCP with serial port not working

Question

I've edited this question after gathering more information. I'm trying to use IOCP for communication through a serial port.

I open the serial port with the overlapped flag:

HANDLE hComm = CreateFile(strPortName,
    GENERIC_READ | GENERIC_WRITE,
    0,
    NULL,
    OPEN_EXISTING,
    FILE_FLAG_OVERLAPPED,
    NULL);

Then I associate "hComm" with the IOCP port.

I'm using ReadFile() to start an overlapped request to read from the serial port. I use ReadFile() like this:

bool bQueued = false;
DWORD dwRead;
BOOL bResult = ReadFile(GetCommHandle(), lpOverlapped->pbBufferData, lpOverlapped->dwBufferSize, &dwRead, (OVERLAPPED*)lpOverlapped);
if (bResult)
{
    // It completed, but will still trigger the completion routine, so don't need to queue another one here.
    bQueued = true;
}
else
{
    DWORD dwError = GetLastError();
    if (ERROR_IO_PENDING == dwError)
    {
        bQueued = true;
    }
    else
    {
        LogQueueReceiveError(lpOverlapped, dwError);

        ResetConnection();
    }
}

The call to ReadFile() always returns immediately with a 'true' result so the IOCP request is queued. However, the operation only completes when the specified number of bytes have arrived at the serial port. In my code, the ReadFile() is called with the size of the receive buffer as the number of bytes to read (which is the way it's done when working with sockets).

If I change the number of bytes to read to a value of 1, then the operation completes as soon as data arrives at the port. Likewise, if I change the number of bytes to read to a value of 8, then the operation completes when the eighth bytes arrives at the port, etc.

If I don't know the number of bytes expected, how can I use IOCP with a serial port without reading a single byte at a time which seems really inefficient?

More information: I've discovered that when I use ReadFile() to initiate the overlapped IO, it does get queued. If I tell it to read 1 byte then it completes when the 1 byte arrives. If I tell it to read, say, 16 bytes - it completes after that many bytes have arrived. I won't know how many bytes to expect beforehand, so I've been calling Readfiile() with the buffer size as the value for that parameter (as seen in most of the MSDN examples for overlapped IO). — Ken, Dec 11 '17 at 18:12
Again more info: When I switch to WaitCommEvent() to initiate the overlapped IO, I get notified when any bytes arrive. However, reading those bytes with ReadFile() results in another overlapped IO being queued, which just results in a death spiral. — Ken, Dec 11 '17 at 18:14
Using WaitCommEvent is the correct way. It ensures that ReadFile is not going to block. Why you are using overlapped I/O for that ReadFile call is not obvious. Don't. — Hans Passant, Dec 11 '17 at 23:18

score 1 · Answer 1 · answered Dec 12 '17 at 00:47

You can use SetCommTimeouts to change this behaviour.

A long time ago (around the year 2000?) I had a similar issue, so I believe the target operating system was Windows NT 4.

My implementation ended up with a bunch of queued single byte reads on an IOCP. While you can set things up using SetCommTimeouts to allow a partial read, at that time you end up paying a timeslice for the timer to give you the partially filled buffer to fill up. For the application this introduced unacceptable latency (10ms or 16ms depending on SMP vs. non-SMP).

The world may have changed since then, or a timeslice of latency might be acceptable to you. Either way, trying out SetCommTimeouts is likely to be useful.

score 1 · Accepted Answer · answered Dec 12 '17 at 10:52

Win32 IOCP with serial port not working

this of course not true. IOCP is perfect working here - when and only when pending io request will be completed or canceled - the packet will be queued to IOCP. so use IOCP in usual way. your issue not in IOCP but in serial driver behavior on read.

I've discovered that ... it completes after that many bytes have arrived.

really this is documented behavior :

Serial.sys continues to transfer bytes until the requested number of bytes are transferred or a time-out event occurs.

or in more detail in SERIAL_TIMEOUTS (equal to COMMTIMEOUTS ):

A read or write request successfully completes when either the specified number of bytes is transferred or the requested read or write operation times out. The request returns the STATUS_SUCCESS status code to indicate that the specified number of bytes was transferred. A read request that exceeds this maximum completes when the time-out occurs, and returns the STATUS_TIMEOUT status code. The Information field of the I/O status block indicates the number of bytes successfully read before the time-out occurred.

so you have 2 choice :

always set nNumberOfBytesToRead to 1 - in this case read request complete as soon as data arrives at the port
set some timeout via SetCommTimeouts or direct send IOCTL_SERIAL_SET_TIMEOUTS control code (this is the same)

possible the best use next:

If both ReadIntervalTimeout and ReadTotalTimeoutMultiplier are set to MAXULONG, and ReadTotalTimeoutConstant is set to a value greater than 0 and less than MAXULONG, a read request behaves as follows:

If there are any bytes in the serial port's input buffer, the read request completes immediately with the bytes that are in the buffer
and returns the STATUS_SUCCESS status code.

If there are no bytes in the input buffer, the serial port waits until a byte arrives, and then immediately completes the read request with the one byte of data and returns the STATUS_SUCCESS status
code.

If no bytes arrive within the time specified by ReadTotalTimeoutConstant, the read request times out, sets the Information field of the I/O status block to zero, and returns the STATUS_TIMEOUT status code.

also need remember that win32 layer almost always lost status which is > 0 (so STATUS_TIMEOUT as well). you got it direct (in FileIOCompletionRoutine callback - the dwErrorCode here really is NTSTATUS code) only if you use BindIoCompletionCallback. if you use CreateThreadpoolIo - you already got IoResult == 0 in IoCompletionCallback on timeout (but will be (NTSTATUS)Overlapped->Internal==STATUS_TIMEOUT). if you use own IOCP and GetQueuedCompletionStatus - again it lost STATUS_TIMEOUT - it simply return you TRUE for completed packet with this code and not set last error. however still will be (NTSTATUS)lpOverlapped->Internal == STATUS_TIMEOUT in this case (not confuse dequeue packet with STATUS_TIMEOUT code and case when GetQueuedCompletionStatus did not dequeue a completion packet because the wait timed out (in this case api return false and last error set to WAIT_TIMEOUT, equal to STATUS_TIMEOUT) )

Win32 IOCP with serial port not working

2 Answers2