Why perror() changes orientation of stream when it is redirected?

Question

The standard says that:

The perror() function shall not change the orientation of the standard error stream.

This is the implementation of perror() in GNU libc.

Following are the tests when stderr is wide-oriented, multibyte-oriented and not oriented, prior to calling perror(). Tests 1) and 2) are OK. The issue is in test 3).

1) stderr is wide-oriented:

#include <stdio.h>
#include <wchar.h>
#include <errno.h>
int main(void)
{
  fwide(stderr, 1);
  errno = EINVAL;
  perror("");
  int x = fwide(stderr, 0);
  printf("fwide: %d\n",x);
  return 0;
}

$ ./a.out
Invalid argument
fwide: 1
$ ./a.out 2>/dev/null
fwide: 1

2) stderr is multibyte-oriented:

#include <stdio.h>
#include <wchar.h>
#include <errno.h>
int main(void)
{
  fwide(stderr, -1);
  errno = EINVAL;
  perror("");
  int x = fwide(stderr, 0);
  printf("fwide: %d\n",x);
  return 0;
}

$ ./a.out
Invalid argument
fwide: -1
$ ./a.out 2>/dev/null
fwide: -1

3) stderr is not oriented:

#include <stdio.h>
#include <wchar.h>
#include <errno.h>
int main(void)
{
  printf("initial fwide: %d\n", fwide(stderr, 0));
  errno = EINVAL;
  perror("");
  int x = fwide(stderr, 0);
  printf("fwide: %d\n", x);
  return 0;
}

$ ./a.out
initial fwide: 0
Invalid argument
fwide: 0
$ ./a.out 2>/dev/null
initial fwide: 0
fwide: -1

Why perror() changes orientation of stream if it is redirected? Is it proper behavior?

How does this code work? What is this __dup trick all about?

Answer 1

TL;DR: Yes, it's a bug in glibc. If you care about it, you should report it.

The quoted requirement that perror not change the stream orientation is in Posix, but does not seem to be required by the C standard itself. However, Posix seems quite insistent that the orientation of stderr not be changed by perror, even if stderr is not yet oriented. XSH 2.5 Standard I/O Streams:

The perror(), psiginfo(), and psignal() functions shall behave as described above for the byte output functions if the stream is already byte-oriented, and shall behave as described above for the wide-character output functions if the stream is already wide-oriented. If the stream has no orientation, they shall behave as described for the byte output functions except that they shall not change the orientation of the stream.

And glibc attempts to implement Posix semantics. Unfortunately, it doesn't quite get it right.

Of course, it is impossible to write to a stream without setting its orientation. So in an attempt to comply with this curious requirement, glibc attempts to make a new stream based on the same fd as stderr, using the code pointed to at the end of the OP:

58    if (__builtin_expect (_IO_fwide (stderr, 0) != 0, 1)
59      || (fd = __fileno (stderr)) == -1
60      || (fd = __dup (fd)) == -1
61      || (fp = fdopen (fd, "w+")) == NULL)
62    { ...

which, stripping out the internal symbols, is essentially equivalent to:

if (fwide (stderr, 0) != 0
    || (fd = fileno (stderr)) == -1
    || (fd = dup (fd)) == -1
    || (fp = fdopen (fd, "w+")) == NULL)
  {
    /* Either stderr has an orientation or the duplication failed,
     * so just write to stderr
     */
    if (fd != -1) close(fd);
    perror_internal(stderr, s, errnum);
  }
else
  {
    /* Write the message to fp instead of stderr */
    perror_internal(fp, s, errnum);
    fclose(fp);
  }

fileno extracts the fd from a standard C library stream. dup takes an fd, duplicates it, and returns the number of the copy. And fdopen creates a standard C library stream from an fd. In short, that doesn't reopen stderr; rather, it creates (or attempts to create) a copy of stderr which can be written to without affecting the orientation of stderr.

Unfortunately, it doesn't work reliably because of the mode:

fp = fdopen(fd, "w+");

That attempts to open a stream which allows both reading and writing. And it will work with the original stderr, which is just a copy of the console fd, originally opened for both reading and writing. But when you bind stderr to some other device with a redirect:

$ ./a.out 2>/dev/null

you are passing the executable an fd opened only for output. And fdopen won't let you get away with that:

The application shall ensure that the mode of the stream as expressed by the mode argument is allowed by the file access mode of the open file description to which fildes refers.

The glibc implementation of fdopen actually checks, and returns NULL with errno set to EINVAL if you specify a mode which requires access rights not available to the fd.

So you could get your test to pass if you redirect stderr for both reading and writing:

$ ./a.out 2<>/dev/null

But what you probably wanted in the first place was to redirect stderr in append mode:

$ ./a.out 2>>/dev/null

and as far as I know, bash does not provide a way to read/append redirect.

I don't know why the glibc code uses "w+" as a mode argument, since it has no intention of reading from stderr. "w" should work fine, although it probably won't preserve append mode, which might have unfortunate consequences.

Answer 2

I'm not sure if there's a good answer to "why" without asking the glibc developers - it may just be a bug - but the POSIX requirement seems to conflict with ISO C, which reads in 7.21.2, ¶4:

Each stream has an orientation. After a stream is associated with an external file, but before any operations are performed on it, the stream is without orientation. Once a wide character input/output function has been applied to a stream without orientation, the stream becomes a wide-oriented stream. Similarly, once a byte input/output function has been applied to a stream without orientation, the stream becomes a byte-oriented stream. Only a call to the freopen function or the fwide function can otherwise alter the orientation of a stream. (A successful call to freopen removes any orientation.)

Further, perror seems to qualify as a "byte I/O function" since it takes a char * and, per 7.21.10.4 ¶2, "writes a sequence of characters".

Since POSIX defers to ISO C in the event of a conflict, there is an argument to be made that the POSIX requirement here is void.

As for the actual examples in the question:

1. Undefined behavior. A byte I/O function is called on a wide-oriented stream.
2. Nothing at all controversial. The orientation was correct for calling perror and did not change as a result of the call.
3. Calling perror oriented the stream to byte orientation. This seems to be required by ISO C but disallowed by POSIX.