2

I have found a difference in behavior between Tcl's [file normalize $path] and the path resolution used by the calls in C, such as open() and realpath().

It revolves around the use of symbolic links immediately followed by ...

The setup: given this directory structure (notice that sidedir_link is a link to a different directory):

/tmp/fileA.txt
/tmp/sidedir/fileA.txt
/tmp/subdir/fileA.txt
/tmp/subdir/sidedir_link -> /tmp/sidedir

the current directory is /tmp/subdir, and I have created the command utils::realpath to be a Tcl call to the C function realpath() in <stdlib.h>.

Here's the transcript. The first two calls show how both resolve a symbolic link. And the last two show the difference in behavior when dealing with the .. that immediately follows the symbolic link.

wish% file normalize sidedir_link/fileA.txt
/tmp/sidedir/fileA.txt
wish% ::utils::realpath sidedir_link/fileA.txt
/tmp/sidedir/fileA.txt
wish% file normalize sidedir_link/../fileA.txt
/tmp/subdir/fileA.txt
wish% ::utils::realpath sidedir_link/../fileA.txt
/tmp/fileA.txt

Note: The difference is /tmp/subdir/fileA.txt versus /tmp/fileA.txt.

Specifically, [file normalize $path] appears to lexically resolve the .. by removing the preceding portion of the path, whereas the C calls actually resolve the symbolic link and then apply the ... If the symbolic link were to /some/location/far/far/away, the resolved path would be /some/location/far/far/fileA.txt.

I couldn't find any mention of this in the Tcl docs, the wiki, or in the bug tracking system.

So, is this a Tcl bug? A C-library bug? Expected behavior?

I'm using Tcl 8.4.9 and Redhat 6 (yes, a bit behind the times...)

Note: I can reproduce the same behavior in Tcl 8.6.1

The reason this is important is that the Tcl/Tk GUI is indicating that a file exists (because it uses the file commands), but when the backend actually opens the file using the C-library call open(), the run fails (because the path is different, and doesn't exist).

Trey Jackson
  • 73,529
  • 11
  • 197
  • 229
  • This looks like an interaction with Tcl's virtual filesystem support. In theory the path could point to a web server path and `realpath()` wouldn't work there, so the code has to work from the textual representation. In fact, the open dialog from Tk could even show you a file inside a database BLOB and your C-library `open()` call will fail as well. So this is maybe expected behaviour, but maybe a bug. There were various bugs found in that part of the code when handling corner cases, so not sure how this one would be classified. – schlenk Jan 29 '14 at 20:11
  • @schlenk I guess you could contort it that way, only that `file normalize` actually does the right thing with symbolic links (using the concrete implementation of the Unix file system) when you do not use `..`. So, it could similarly use the concrete implementation to resolve the `..`. For example, I could have a database BLOB where `..` is a valid file name (and doesn't refer to the parent directory), in which case the Tcl implementation would appear to never work... – Trey Jackson Jan 29 '14 at 20:36
  • Not sure you know the Tcl vfs system, so the blob example might be bad. But to get back to the point, there are some tests about `file normalize` and `..` in the path (see tests `filesystem-1.*` in https://core.tcl.tk/tcl/artifact/0a3fce191778e80a). I didn't see your exact case in there, so maybe its a missed corner case. – schlenk Jan 29 '14 at 23:07
  • I'll check those out. It's a bit hard to parse w/out using the interpreter, but they do have a bunch with .. – Trey Jackson Jan 29 '14 at 23:17
  • It wouldn't be the first time there's been a bug in `file normalize`, but we can't just use `realpath()` as that doesn't know about Tcl's VFS layer. (I don't try to fix problems in this subsystem; it scares me.) – Donal Fellows Jan 29 '14 at 23:48
  • @DonalFellows FWIW, I did look more closely at the file tests shipped with Tcl 8.4, and they didn't test this situation. – Trey Jackson Feb 13 '14 at 16:44
  • @TreyJackson I wouldn't be at all surprised. I really don't want to maintain the VFS code though; it's really tricky in places and has all sorts of subtleties that are only important on some platforms (or with a third party VFS “mounted”). I save my attention for other parts of Tcl… – Donal Fellows Feb 14 '14 at 20:44
  • @DonalFellows understood. I ended up overriding `[file]` and `[open]` to catch this condition and DTRT (at least as I see it). Intentionally skipping the other possible commands like `[glob]` that also deal with file paths. It's such a corner case... – Trey Jackson Feb 14 '14 at 20:58

1 Answers1

1

This smells a lot like a bug to me — given what I remember of the history of the pathname resolution and normalization code, I wouldn't be at all surprised — so I've made Bug Ticket #6c49da8a19. I don't maintain this part of Tcl so I don't know even how to begin to address it other than by filing a bug report. If you want to provide further information, please do so via our issue tracker.

Donal Fellows
  • 133,037
  • 18
  • 149
  • 215