12

The standard library function fopen is declared in <stdio.h> as:

FILE *fopen(const char * restrict filename, const char * restrict mode);

This is also how the function prototype appears in the C Standard.

Why are the arguments restrict qualified?

chqrlie
  • 131,814
  • 10
  • 121
  • 189
  • 1
    See https://en.wikipedia.org/wiki/Restrict. In short, it means you are not allowed to have `filename` and `mode` point to the same string. – Nate Eldredge Feb 13 '16 at 23:14
  • Good question. There doesn't seem to be a valid reason to prevent aliasing in this case. Maybe for the sake of consistency (all other functions in stdio.h using this qualifier)? – michaelmeyer Feb 13 '16 at 23:18
  • 2
    @NateEldredge: I don't believe it means that in a function prototype, and it would be a ridiculous restriction, `fopen` is not even supposed to modify these strings. Why is wrong with `fopen("r", "r")` ? – chqrlie Feb 13 '16 at 23:18
  • 1
    It seems to me that `fopen("r", "r")` would be legal, but the compiler would have to avoid folding them into the same location, so that they are not the same string (at the same address) but two different strings with the same data. What I think would be illegal is `char *s = "r"; fopen(s,s);`. I think you're also not allowed to have them point into the same string. I agree it doesn't seem to make sense given that they are `const`. – Nate Eldredge Feb 13 '16 at 23:23
  • @michaelmeyer: `rename` is declared in `` as `int rename(const char *old, const char *new);` – chqrlie Feb 13 '16 at 23:23
  • @NateEldredge: But why would `char *s = "r"; fopen(s,s);` be illegal? The restriction you are implying is not even mentioned in the function description in the Standard C11 7.21.5.3. The compiler will share the string literals too. – chqrlie Feb 13 '16 at 23:26
  • 3
    @chqrlie: It would be illegal because that is what `restrict` **means**. – Nate Eldredge Feb 13 '16 at 23:27
  • @NateEldredge: `memcpy(ptr, ptr, 0);` is legal too. `restrict` does not mean what you imply. – chqrlie Feb 13 '16 at 23:38
  • Hmm, maybe I need to ask a new question about `restrict`. – Nate Eldredge Feb 13 '16 at 23:40
  • 1
    `memcpy(ptr, ptr, 0);` is illegal. (the constraint violation) – BLUEPIXY Feb 13 '16 at 23:45
  • Read the restrict reference that I posted in an answer below, @NateEldredge. It may help. – David Hoelzer Feb 13 '16 at 23:50
  • 1
    @BLUEPIXY: C11 7.24.2.1: *The memcpy function copies n characters from the object pointed to by s2 into the object pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined.* both objects have a size of `0`, they do not *overlap*. – chqrlie Feb 13 '16 at 23:50
  • 1
    The Standard is all about language and conciseness `;-)` – chqrlie Feb 13 '16 at 23:52
  • @chqrlie Specified restrict is irrelevant to whether n is 0. – BLUEPIXY Feb 13 '16 at 23:57
  • @BLUEPIXY: are you implying that `memcpy` between non overlapping sections of the same array invokes undefined behavior? – chqrlie Feb 14 '16 at 00:07
  • @chqrlie It is not dependent on the actual operation of the internal of function when I understand. – BLUEPIXY Feb 14 '16 at 01:01
  • Another find: https://groups.google.com/forum/#!topic/comp.std.c/J4Zq862FvWk . Seems like chqrlie already discussed this quite extensively 8 years ago. – Ctx Feb 14 '16 at 01:05
  • @Ctx: I did indeed! I had almost forgotten about that. Larry Jones gave the final word. The committee intended for `restrict` to mean some implied restriction on overlapping objects, which in the case of `fopen` does not seem to make sense. I should have posted on stackoverflow then, just two months after inception. – chqrlie Feb 14 '16 at 01:26
  • @BLUEPIXY: *au contraire*: what you are indicating may have been the committee's intent, as can be derived from Larry Jones' posts in the comp.std.c discussion, but the formal definition of `restrict` only pertains to the actual operation of the internals of the function, where it is just a promise from the programmer to the compiler. – chqrlie Feb 14 '16 at 09:44
  • @BLUEPIXY: "*`memcpy(ptr, ptr, 0); is illegal. (the constraint violation)`*" -- No, it's not a constraint violation. (If it is, please cite the relevant constraint in the standard.) It's probably undefined behavior, assuming that an object "overlaps" itself. – Keith Thompson Jul 27 '16 at 01:43
  • @KeithThompson: the standard says: *The memcpy function copies n characters from the object pointed to by s2 into the object pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined.* The function copies no characters -> copying does not take place -> the last phrase does not apply. Behavior is not undefined. – chqrlie Jul 27 '16 at 01:54
  • @chqrlie: Touché! (I probably shouldn't have stopped reading after the second argument.) – Keith Thompson Jul 27 '16 at 02:01
  • @KeithThompson: What byte would be accessed via both pointers? I recognize that `memcpy(ptr,ptr,n)` would be allowed to yield wacky behavior in cases where `n` is non-zero,. though I see no purpose outside of sanitizing builds whose purpose is to guard programmers from deliberately-obtuse compilers. I see no legitimate basis for wackiness in the `n=0` case, however. – supercat Jul 27 '16 at 02:47
  • @supercat: I already acknowledged chrqlie's point; `memcpy(ptr, ptr, 0)` has, I think, well defined behavior (as long as `ptr` is valid). – Keith Thompson Jul 27 '16 at 04:16
  • @KeithThompson: I agree that `ptr` should be valid, but I think it could be null and still not invoke undefined behavior. Similarly, `ptr` could point just past the end of a valid array. Both cases of pointer values that are valid for passing and comparing but invalid to dereference. – chqrlie Jul 27 '16 at 08:44
  • @chqrlie: See [N1570](http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf) 7.24.1p2 and 7.1.4p1. The behavior of `memcpy(NULL, NULL, 0)` is undefined. – Keith Thompson Jul 27 '16 at 15:56
  • @KeithThompson: a sad and unnecessary limitation. Whether a pointer to the end of the array can be passed when `n` is `0` remains unclear: it is not strictly outside the address space as its value can be validly compared to another pointer into the same array, and *the pointer has a value such that all address computations and accesses to objects (that would be valid if the pointer did point to the first element of such an array) are in fact valid* since there are none. – chqrlie Jul 27 '16 at 16:06

3 Answers3

2

There does not seem to be any compelling reason for fopen arguments to be restrict qualified in the prototype in <stdio.h>.

restrict qualifying a pointer is a promise by the programmer that the object pointed to by said pointer will only be accessed via this and any other pointer based on it within the given scope.

In a function prototype, such a promise is moot.

Nate Eldredge offered the explanation that it means you are not allowed to have filename and mode point to the same string. But this claim seems irrelevant and such a constraint is unneeded and not mentioned in the definition of fopen in section 7.21.5.3 of the C Standard.

The prototype for setbuf has the same restrict qualifier for its arguments:

void setbuf(FILE * restrict stream, char * restrict buf);

I can understand why an implementer would qualify stream and buf with the restrict keyword to tell the compiler that a modification to the FILE structure during the scope of setbuf has no effect on the contents of buf and vice versa.

The same might be true of an implementation of fopen where the programmer tells the compiler that the FILE structure manipulated by fopen does not overlap the filename, nor the mode. But qualifying both filename and mode is a false promise since it implies a constraint that is not present in the Standard.

My conclusion is that restrict qualifying arguments in function declaration prototypes is unneeded and deceptive. It reduces readability and induces false interpretations.

chqrlie
  • 131,814
  • 10
  • 121
  • 189
  • There are many things about stdio that don't make much sense and could be done better and safer. – Petr Skocik Feb 13 '16 at 23:36
  • @PSkocik: Given the proficiency of the people crafting the Standard and the effort they put into it, there must be a deeper reason than this. – chqrlie Feb 13 '16 at 23:41
  • 1
    So maybe I am misunderstanding how `restrict` works. But it seems to me that if `fopen` is defined using `restrict`, this is a commitment by the programmer that the object pointed to by `mode` will not be accessed through any other pointer during the lifetime of `fopen`. If you called `char *s = "r"; fopen(s,s);`, this commitment would be violated. Therefore, you must not do that. I see the `restrict` in the prototype as an indication of that, and perhaps a hint to help the compiler detect such code if it can. – Nate Eldredge Feb 13 '16 at 23:49
  • @NateEldredge: What you are writing makes sense. But such a restriction for `fopen` does not and is not written in the Standard. Hence my question. – chqrlie Feb 13 '16 at 23:55
  • My view is that this restriction is implied by the use of `restrict` in the Standard's prototype! But I agree with you in that I don't understand why it was put there. – Nate Eldredge Feb 13 '16 at 23:56
  • @ctx: that's a bummer - that site aggresively reverts to a 'mobile' view on my iPad and on that, the page gets a 404. Care to repeat the salient points in an answer? (Also in the vein of "off site resources may become unavailable". Of course I could walk over to my desktop.) – Jongware Feb 14 '16 at 00:24
  • @Ctx: interesting article, but not really compelling. Even the C99 rationale http://www.open-std.org/jtc1/sc22/wg14/www/docs/n937.pdf is vague about the use of `restrict` in function declaration prototypes. For `memcpy` it says: *The restrict qualifier can be used to express the restriction on overlap in a new prototype that is compatible with the original version:*. This hardly applies to `fopen`. – chqrlie Feb 14 '16 at 00:25
  • 1
    @Jongware Citation: The name of the file and its open mode are accessed through unique pointers in fopen(). Therefore, it's possible to preload the values to which the pointers are bound. Indeed, the C99 standard revised the prototype of the function fopen() – Ctx Feb 14 '16 at 00:32
  • @Ctx: `fopen` does not modify the arrays pointed to by `filename` and `mode`, preloading the values is an implementation choice not mandated by the Standard, and the fact that both arrays can overlap does not prevent such preloading anyway. – chqrlie Feb 14 '16 at 00:42
  • 1
    @chqrlie You probably just do not quite understand what this means. FWIW, I do not fully understand it either. But I think it is a bit too easy to say, that all other people are wrong (those, who added the restrict keyword to fopen() and those trying to explain, why). – Ctx Feb 14 '16 at 00:46
  • @chqrlie I don't understand why you're saying that using restrict in a function prototype is moot, and deceptive. It's the main use for restrict to tell both the callers of a function, and help code generation in the function implementation, to state that one pointer argument cannot alias another argument.. The constraint is also explicitly specified for fopen() in the standard, as 7.21.5.3 gives the prototype for fopen() which includes the restrict qualifier.. – nos Feb 14 '16 at 02:21
  • @nos: the formal definition of `restrict` does not apply to function prototype declarations, so while it does help code generation in the implementation, it does not formally tell anything to the caller of a function. The constraint you mention for `fopen` is not explicitly specified in the Standard and does not really make sense to other readers. – chqrlie Feb 14 '16 at 02:28
  • @chqrlie And I'm asking you why you think that restrict does not apply to function prototype declarations, it tells the caller explicitly that the implementation expects the pointers to not point at the same thing. The fopen function is documented in the standard(C99 and above) as `FILE *fopen(const char * restrict filename, const char * restrict mode);` and that's as explicit and authoritative as any english text there. – nos Feb 14 '16 at 02:33
  • @nos: John Bollinger's answer is quite clear as to why `restrict` does not apply to function prototype declarations. Furthermore, `restrict` does not mean *that the implementation expects the pointers to not point at the same thing*, it tells the compiler that objects accessed through one restrict qualified pointer in actual code are not also accessed through another restrict qualified pointer. The implementation of `fopen` could conceivably special case `filename == mode` and honor the promise without the useless and probably unintended restriction on `filename` and `mode` possible overlap. – chqrlie Feb 14 '16 at 08:13
  • @chqrlie Well, I can't agree with any of that, except the last point where it's probably quite useless for fopen() to have this restriction. – nos Feb 14 '16 at 12:05
2

The formal definition of the meaning of restrict is given in section 6.7.3.1 of C2011. Note in particular that elements of the parameter list of a function prototype that is not part of a function definition do not meet the conditions set forth in paragraph 1 of that section:

Let D be a declaration of an ordinary identifier that provides a means of designating an object P as a restrict-qualified pointer to type T.

The parameter list of such a prototype does not provide any means to make such a designation. Therefore, nothing in the section gives any direct effect to a restrict qualifier in that context.

The best interpretation is probably as a notice to users of the function that the parameters are declared with the restrict qualifier in the function definition, where that qualifier does have effect.

Note also, however, that restrict is not a blanket promise of absence of aliasing. Instead it is a qualified promise that if the target of the restrict-qualified pointer is modified in any way then it will be accessed only via that pointer, within the scope of the pointer.

Coming back around to fopen(), then, the restrict qualifiers in the function prototype have nothing to say about the definedness of the behavior of any call of the function. That is, this is not inherently undefined:

char s[] = "r";
FILE *f = fopen(s, s);

The execution of the function is another story -- or it would be if the pointer targets were not also const-qualified. Supposing that the prototype expresses the effective qualifiers for the function's definition, if the two arguments alias each other and if their target were modified by the function, then the function would invoke undefined behavior if it accessed the target via the other pointer. Inasmuch as the const qualification of the pointer targets means that the function would invoke UB by modifying the target in the first place, the restrict qualification is moot.

The execution of the function is another story. Although the const qualification of the parameter targets means that we should assume that fopen() will not attempt to modify them via those pointers, the targets themselves are not necessarily const. They could conceivably be modified outside fopen(), via a non-const lvalue, and that would still be relevant to restrict qualification. Suppose that the prototype expresses the effective qualifiers for the function's definition. If the two arguments alias each other, and if their shared target is modified anywhere in the program, then fopen() would invoke UB when it accessed the target via both pointers.

John Bollinger
  • 160,171
  • 8
  • 81
  • 157
  • Thank you for a detailed explanation... regarding `fopen`, is your conclusion the same as mine, namely that `restrict` qualifying the arguments serves no definite purpose? – chqrlie Feb 14 '16 at 00:48
  • @chqrlie, I'd say it differently: that `restrict`-qualification in a function prototype has no associated semantics. In general, however, the author of such a prototype may have had a specific purpose for the `restrict`-qualification. And in particular, the authors of the language standard surely had at least one specific purpose for the `restrict` qualifiers in the function prototypes that appear in the standard. – John Bollinger Feb 14 '16 at 03:40
  • I completely agree with you, but in the case of `fopen`, I'd say this specific purpose is unclear. – chqrlie Feb 14 '16 at 08:00
  • @chqrlie, I do not dispute that the purpose in this case is unclear, but I think a reasonable interpretation is that the prototype reflects the qualifiers that callers must assume are associated with the actual definition. Inasmuch as `restrict` has implications not just on modifications but on *accesses* via `restrict`ed pointers, such advice may be warranted. I have revised my last paragraph to speak to this issue. – John Bollinger Feb 15 '16 at 16:10
  • Since `fopen` does not modify either argument, the only time `restrict` would be an issue would be if an application passed a pointer to something that would get modified during the execution of the `fopen` function itself (e.g. an open-files table). The Standard doesn't define any means via which an application could get a pointer to such a thing, but some implementations might. – supercat Jul 27 '16 at 02:50
1

It amounts to a promise that the data will not be changed. There is no actual enforcement of this promise, as is often the case in C, but failing to live by this promise can result in undefined behavior.

Essentially, the restrict qualifier there means that no other pointer will change the value of the filename or of the access mode while the file handle is valid. Here's an excerpt from a website that covers this that is quite good:

Restrict is a "no data hazards will be generated" contract between the programmer and the compiler. The compiler relies on this information to make optimizations. If the data is, in fact, aliased, the results are undefined and a programmer should not expect the compiler to output a warning. The compiler assumes the programmer is not lying.

So, why is it there? Because, when the fopen library function was written, the writer decided to ask that you not change the strings that you pass it after you ask it to open a file. To be frank, I can't see why this would even be requested because, to my knowledge, once the file is open only the file descriptor matters and the file name and mode are never referred to again internally.

David Hoelzer
  • 15,862
  • 4
  • 48
  • 67
  • I'm afraid your explanation does not hold. The `FILE*` can be used long after the filename has gone out of scope, no such restriction is mentioned in the Standard. It is quite common for `fopen` to be called with a filename stored in an array with automatic storage. – chqrlie Feb 13 '16 at 23:35
  • 1
    `const char *filename` is the promise that `fopen` will not modify the filename, nothing else is promised. – chqrlie Feb 13 '16 at 23:36
  • 1
    Well, I'm explaining it in the context of what `restrict` means. Frankly, I can't see any reason that it matters since the FD is the only thing that *really* matters internally.. The file name is never referenced again. – David Hoelzer Feb 13 '16 at 23:40
  • Then this statement is not true: *Because the fopen library function requires that you not change the strings that you pass it after you ask it to open a file* – chqrlie Feb 13 '16 at 23:43
  • Good call. Updated to reflect the comments I made. – David Hoelzer Feb 13 '16 at 23:45
  • 2
    I agree with your final point: *I can't see why this would even be requested*. I'm going one step further and pretend that this was not even the writer's intent nor the actual meaning of the `restrict` keyword in this context. – chqrlie Feb 13 '16 at 23:47
  • I sort of wonder if, at some point in time, it *was* required in some iteration of the code, but later that change. I know that I can relate to putting certain things into an API that later became deprecated even though I never updated the header file. – David Hoelzer Feb 13 '16 at 23:48
  • The standard seems to speak of all *accesses* to the object, not just all *changes*. My reading suggests that `restrict` imposes a contract even on read-only data. – Nate Eldredge Feb 13 '16 at 23:54
  • 1
    I don't think you're wrong, Nate. It's a rarely used qualifier, in my experience. Again, I think that it's an artifact of antiquity in this particular case. – David Hoelzer Feb 13 '16 at 23:55