existing files are "invisible" to some applications

Question

I have a problem with accessing files on an ext4 file system mounted as type NFS on a Linux server running CENTOS 6.5. I know the files exist on this file system because I can see them when I explicitly call them (e.g. with ls, awk, or cat), but are not visible to some programs like rsync or when piped from ls into another program like grep or awk.

As an example to clarify, the following will return empty:

ls /mnt/seisvault2/data/sac/201402/20140203_220000_MAN/ | grep WCI\.BH1\.IU.10

While this shows me that the file actually exists:

ls /mnt/seisvault2/data/sac/201402/20140203_220000_MAN/WCI.BH1.IU.10.

In the numerous directories where this is a problem, most files appear normally, without needing to be called explicitly as shown in the example.

Can anyone help me understand this problem?

As an example of why this is a problem, rsync -a copies the "missing" files on its first run, but for each subsequent run, it doesn't think the file is in the target directory, and so copies it again.

Output of `ls -ld /mnt/seisvault2/data/sac/201402/20140203_220000_MAN` please. — Mark Wagner, Jun 20 '14 at 23:06
Output of the ls -ls command: `drwxrwxrwx. 2 nvca223 seismic 20480 Feb 3 17:55 /mnt/seisvault2/data/sac/201402/20140203_220000_MAN` — geoweaser, Jun 21 '14 at 13:23
And, I think it's NFSv3: `yum --version nfs` returns `3.2.29` — geoweaser, Jun 21 '14 at 13:23
To get some insight you could try `ltrace ls /mnt/seisvault2/data/sac/201402/20140203_220000_MAN/` and `strace ls /mnt/seisvault2/data/sac/201402/20140203_220000_MAN/` and look for errors like `EPERM` or `EACCESS`. Also look of there is any mention of WCI.BH1.IU.10. — Mark Wagner, Jun 23 '14 at 18:34
First, thanks for your help, Mark. I executed the ltrace and strace calls you gave, and did not see EPERM or EACCESS errors. Also, there is no mention of WCI.BH1.IU.10 in the outputs. However, as before, when I append WCI.BH1.IU.10 onto the ltrace and strace calls, it is discovered with no errors. — geoweaser, Jun 27 '14 at 18:35
How many files exist in this /mnt.....MAN directory? Is it possible there are more than the Bash pipe buffer can support? — Daniel, Feb 06 '15 at 18:19
Good question. There are not too many files: on the order of 100. And I'm using tcsh, but I don't think that matters. — geoweaser, Feb 06 '15 at 18:53

score 1 · Accepted Answer · answered Dec 14 '15 at 21:18

1

This problem seems to be resolved by a configuration setting under File Sharing / NFS. Originally, I had Operation Mode configured as "User Mode." However, it appears this should be "Kernel Mode."

answered Dec 14 '15 at 21:18

geoweaser

63
1
1
7

score 0 · Answer 2 · 2014-06-20T20:59:51.173

Grep is using regular expressions, of which, '.' happens to be a special character. To demonstrate, imagine I make an empty and do the following:

[root@db test]# touch testfile{1,2,3,4,5}
[root@db test]# touch testfile{1,2,3,4,5}.other
[root@db test]# ls -lah
total 8.0K
drwxr-xr-x  2 root root 4.0K Jun 21 00:43 .
dr-xr-x---. 4 root root 4.0K Jun 21 00:41 ..
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile1
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile1.other
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile2
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile2.other
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile3
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile3.other
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile4
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile4.other
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile5
-rw-r--r--  1 root root    0 Jun 21 00:43 testfile5.other

Now, lets do some grepping! As expected, ls | grep test returns all the files in the directory:

[root@db test]# ls | grep test
testfile1
testfile1.other
testfile2
testfile2.other
testfile3
testfile3.other
testfile4
testfile4.other
testfile5
testfile5.other

But lets say we just wanted the files with a '.' in them:

[root@db test]# ls | grep .
testfile1
testfile1.other
testfile2
testfile2.other
testfile3
testfile3.other
testfile4
testfile4.other
testfile5
testfile5.other

Huh, well that sucks. As mentioned above, '.' has special meaning in regular expressions (namely, match 1 character). How to solve this problem? Backslash comes to the rescue...as the... ESCAPANATOR

[root@db test]# ls | grep '\.'
testfile1.other
testfile2.other
testfile3.other
testfile4.other
testfile5.other

Note that I've put it in single quotes. This will work with double quotes as well -- in most situations (shell expansion can burn you on more complex regexes). If you need to do it without any quotes, you have to escape twice (once for bash, once for grep). It looks like this:

[root@db test]# ls | grep \\.
testfile1.other
testfile2.other
testfile3.other
testfile4.other
testfile5.other

Thanks for the grep advice. But, this doesn't help me understand the problem. For example, rsync doesn't see the files either. The following should certainly find the file, but it doesn't: `ls /mnt/seisvault2/data/sac/201402/20140203_220000_MAN/ | grep WCI | grep BH1 | grep 10` As with the previous call I showed, this does not return the file. — geoweaser, Jun 21 '14 at 13:26

existing files are "invisible" to some applications

2 Answers2