21

Both Linux and the GNU userspace (glibc) seem to have a number of "WONTFIX" bugs, i.e. bugs which the responsible parties have declared their unwillingness to fix despite clearly violating the requirements of ISO C and/or POSIX, but I'm unaware of any resource for programmers which lists such bugs and suggestions for working around them.

Here are a few that come to mind:

  • The Linux UDP select bug: select (and related interfaces) flag a UDP socket file descriptor ready for reading as soon as a packet has been received, without confirming the checksum. On subsequent recv/read/etc., if the checksum was invalid, the call will block. Working around this requires always setting UDP sockets to non-blocking mode and dealing with the EWOULDBLOCK condition. If I remember correctly, MaraDNS was the first notable project affected by this bug and the first to complain (unsuccessfully) to have it fixed. Note: As pointed out by Martin v. Löwis, apparently this bug has since been fixed. Workarounds are probably only necessary if you need to support really outdated versions of Linux.
  • The printf family in the GNU C library wrongly treats arguments to %s as multibyte character strings instead of byte strings when a field precision (as in %.3s) is specified, potentially causing truncated output. I know of no workaround except replacing the whole printf subsystem (or simply not using the printf family of functions with non-multibyte-character byte strings, but this can be problematic if you want to process legacy-codepage strings using snprintf while in a UTF-8 locale).
  • Wrong errno result codes for certain syscalls (can't remember which ones right off). Usually these are easy enough to check for if you just read the GNU/Linux man pages and compare them to the standard. (I cannot find the references for this and perhaps I am mistaken. The closest I can find is the issue of ENOTSUP and EOPNOTSUP having the same value; see PDTR 24715.

What are some more bugs and workarounds we can add to this list? My goals in asking this question are:

  1. To build a more complete list of such bugs so that both new and experienced programmers can quickly become aware of potential issues that could arise when running an intended-to-be-portable program on GNU/Linux.
  2. To leverage the SO collective brain to think up clever and unobtrusive standard workarounds for as many such bugs as possible, instead of everyone having to invent their own workarounds after getting stung, and possibly doing so in suboptimal, ugly, or hackish ways - or worse yet, in ways that break support for more-conformant systems.
R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • 7
    Whoever voted to close, please explain. I don't see how this could be off-topic since I asked about workarounds which are clearly a programming topic. I admit it is mildly argumentative which would be a separate reason to propose closing, but if the focus is a constructive search for solutions that deal with these problems in unobtrusive ways rather than a mere blamefest, I think it's a worthwhile question. – R.. GitHub STOP HELPING ICE Dec 07 '10 at 20:59
  • 3
    What bugs? If Linux is not standards-compliant, it's clearly the standard that's wrong. – Franci Penov Dec 07 '10 at 21:01
  • @R.. - just kidding. but wouldn't be surprised if there are people that actually think this way. And for what it's worth - I think this is a good question. – Franci Penov Dec 07 '10 at 21:02
  • 5
    SO needs distinct +1 funny and +1 informative votes for comments... :-) – R.. GitHub STOP HELPING ICE Dec 07 '10 at 21:02
  • You obviously know what WONTFIX is, and what remains of your question is how to work around them... in general? Or just the three specific bugs you mention, of which two you've already answered yourself? – Stéphan Kochen Dec 07 '10 at 21:06
  • 2
    This sounds like a good candidate for a community wiki. – Ape-inago Dec 07 '10 at 21:11
  • @Shtééf: Added some details on my goals. @Ape-inago: Does community-wiki still exist? I thought it was removed... – R.. GitHub STOP HELPING ICE Dec 07 '10 at 21:12
  • I flagged the question with a comment that it should be CW. Hopefully a mod will see. – Gabe Dec 07 '10 at 21:35
  • Can you please either clarify or remove your claim about incorrect errno values? Such an unspecific claim is indeed argumentative; to discuss it (and react to it in software), one would need to know which system call specifically might report what specific incorrect errno code. – Martin v. Löwis Dec 07 '10 at 21:41
  • 10
    I'm not sure this question is a good use of the SO system. It's pretty broad, somewhat subjective, and there's no way to say what answers are better or worse than others. SO is not really good for collecting "list of X". For the first point, about developing a list of bugs, a wiki or bug tracker would be better suited than SO. As for leveraging the "SO brain" to find workarounds, it would be much better to have one question per bug, so workarounds could be voted on and accepted, not trying to cram the bugs and their workarounds into answers and/or comments. – Brian Campbell Dec 07 '10 at 22:42
  • @Brian: I agree the SO system doesn't work so well for this, but I don't know anything better. A bugtracker would be nice if the goal were to get such bugs fixed, but SO seems to be the best place to put a resource for programmers (if for no other reason than its extremely high pagerank on Google). – R.. GitHub STOP HELPING ICE Dec 07 '10 at 22:46
  • @R: I suspect the best approach would be a single question for each such issue identified, grouped by a tag. The identification of such an issue and the best workarounds are quite possibly going to be contributed by different people. – caf Dec 08 '10 at 03:00

2 Answers2

5

I can't reproduce the printf issue that you claim. Running the program

#include <stdio.h>
#include <locale.h>

int main()
{
        setlocale(LC_ALL, "");
        printf("%.4s\n", "Löwis");
        return 0;
}

in a de_DE.UTF-8 locale prints "Löw", which looks right to me: I asked for 4 bytes, and got four bytes (ö is 2 bytes). Had the library counted multi-byte characters, the output should have been "Löwi". This is with glibc 2.11.2.

Edit: Changing the string to "%.2s\n" will just print "L", i.e. only one byte. However, this is conforming to the specification, which says

If the precision is specified, no more than that many bytes shall be written.

(emphasis mine), and then

In no case shall a partial character be written.

So since printing two bytes (i.e. the L, and the lead byte of ö) would result in a partial character being written, it would be non-conforming to print incomplete UTF-8.

Martin v. Löwis
  • 124,830
  • 17
  • 198
  • 235
  • Nice find! But you're focusing on his examples instead of his actual question: how to work around the bugs. Going after each example one by one to prove they're not bugs will just cause him to spew out more examples. – Konerak Dec 07 '10 at 21:52
  • 3
    Try changing it to `"xxxö"` and you'll see that you get just 3 bytes. This is a dumb example, but a real example would be if you were trying to work with strings in multiple different encodings that don't match the locale's encoding. ISO C allows you to do this (because `%s` is specified purely in terms of bytes) but glibc forbids it. Anyway I don't want to argue over whether these things are bugs. They're already well-documented by well-qualified people. I'm asking about building a list of problems and workarounds. – R.. GitHub STOP HELPING ICE Dec 07 '10 at 22:13
  • 1
    Or just change the format string to `"%.2s"` to see the bug. – R.. GitHub STOP HELPING ICE Dec 07 '10 at 22:14
  • See my edit. It is still conforming. – Martin v. Löwis Dec 07 '10 at 22:21
  • 1
    @R..: If you are interested in building lists of bugs and work-arounds, you should consequentially remove things from the lists that turn out not be bugs. Claiming that things are bugs that actually aren't is subjective and argumentative. – Martin v. Löwis Dec 07 '10 at 22:23
  • 1
    @Martin: Regarding your edit, you're reading the text about the `%ls` format specifier. "In no case shall a partial character be written." refers to what happens when a `wchar_t` string is converted to a multibyte string. Regarding plain `%s`: "Bytes from the array shall be written up to (but not including) any terminating null byte." Note the **shall be written**. There is no option for the implementation to exclude some of them based on semantic interpretation. – R.. GitHub STOP HELPING ICE Dec 07 '10 at 22:34
  • By the way, if I remember correctly, you'll find details of the decision to make `%s` deal purely with byte strings in the ISO C rationale document. – R.. GitHub STOP HELPING ICE Dec 07 '10 at 22:36
  • In the copy of the rationale at http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf, I couldn't find any discussion of %s. – Martin v. Löwis Dec 07 '10 at 22:48
  • Martin appears to be technically correct - the wording appears to allow a degenerate implementation to print no characters at all whenever a precision is used with `%s`. – caf Dec 08 '10 at 03:06
  • @caf: you read the wrong paragraph. The wording specifies exactly what happens. There is nothing left up to the implementation. – R.. GitHub STOP HELPING ICE Dec 08 '10 at 04:32
  • @R: The wording I am reading is just that for `%s`. The phrase *"Bytes from the array shall be written up to..."* is obviously *meant* to imply "all bytes, unless the precision is reached first", but a literal reading seems to allow for "some number of bytes". – caf Dec 08 '10 at 05:03
4

I don't believe that the UDP issue actually exists. In the current Linux kernel, udp_poll reads

/**
 *      udp_poll - wait for a UDP event.
 *      @file - file struct
 *      @sock - socket
 *      @wait - poll table
 *
 *      This is same as datagram poll, except for the special case of
 *      blocking sockets. If application is using a blocking fd
 *      and a packet with checksum error is in the queue;
 *      then it could get return from select indicating data available
 *      but then block when reading it. Add special case code
 *      to work around these arguably broken applications.
 */
unsigned int udp_poll(struct file *file, struct socket *sock, poll_table *wait)
{
        unsigned int mask = datagram_poll(file, sock, wait);
        struct sock *sk = sock->sk;

        /* Check for false positives due to checksum errors */
        if ((mask & POLLRDNORM) && !(file->f_flags & O_NONBLOCK) &&
            !(sk->sk_shutdown & RCV_SHUTDOWN) && !first_packet_length(sk))
                mask &= ~(POLLIN | POLLRDNORM);

        return mask;

}

So it seems to me that it does hide UDP packets with bad checksums from being reported through select/poll. This version of the code is being used since revision 85584672 (2009). But even before (since at least 2005), the code apparently was doing the same kind of dropping bad packets in select/poll already.

Jens
  • 69,818
  • 15
  • 125
  • 179
Martin v. Löwis
  • 124,830
  • 17
  • 198
  • 235
  • Nice find! But you're focusing on his examples instead of his actual question: how to work around the bugs. Going after each example one by one to prove they're not bugs will just cause him to spew out more examples. – Konerak Dec 07 '10 at 21:51
  • 3
    But that's the claimed objective of this article: find specific issues, and post specific work-arounds. So for the first two problems, it turns out that no work-around is needed (atleast with the current versions). So anybody believing that these are actual problem is being helped by knowing that they are not. If real bugs get presented, then we could start discussing work-arounds. – Martin v. Löwis Dec 07 '10 at 22:13
  • You're right on one. I listed it as a WONTFIX because that's what it originally was. Apparently enough people eventually complained (or maybe the kernel internals changed enough that fixing it became easy) that it was actually fixed. Thanks for the find. As for the other, you've misread the standard. – R.. GitHub STOP HELPING ICE Dec 07 '10 at 22:41