5

Trying to get a list of filenames from a directory that is mounted to a CIFS file system. When there are a certain number of files (>55) that are in a sequence, i.e. 00000.xxx through 00299.xxx, some files are missing from the list (e.g. file 00055.xxx might not be there). Doesn't matter the size of the files. They can be zero sized. Using the same code from a local file system works just fine. The missing file(s) seem to be consistent with the number of files in the directory (e.g. 56 files in the directory might be missing file 21 always, whereas 300 files might be missing 55, 81, 137). No errors are returned in the below programs.

This is running on CentOS 7.4.

Reduced the golang program to two different methods (both fail):

Method 1:

package main
import "os"
import "fmt"
import "io/ioutil"

func main() {
    directory := os.Args[1]

    files, readDirError := ioutil.ReadDir(directory)

    if readDirError != nil {
        fmt.Printf("Error in readDir\n")
        fmt.Println(readDirError)
        fmt.Println("\n")
    }

    for x, f := range files {
        fmt.Printf("%d => %s\n", x, f.Name())
    }
}

Method #2

package main
import "os"
import "fmt"

func main() {
    dir, readDirError := os.Open(os.Args[1])

    defer dir.Close()

    if readDirError != nil {
        fmt.Printf("Error in readDir\n")
        fmt.Println(readDirError)
        fmt.Println("\n")
    }

    fi, err := dir.Stat()

    if err != nil {
        fmt.Println(err)
        return
    }

    if fi.IsDir() {
        fis, err := dir.Readdir(-1)

        if err != nil {
            fmt.Println(err)
            return
        }

        for _, fileinfo := range fis {
            fmt.Printf("%s\n", fileinfo.Name())
        }
    }
}

Created a C program that did the same thing, and this works every time, i.e. it lists all the files in the directory. So, it doesn't seem to be a problem with getting all the information from the file system.

#include <dirent.h>
#include <stdio.h>

int main(int c, char **argv)
{
  DIR           *d;
  struct dirent *dir;
  d = opendir(argv[1]);
  if (d)
  {
    while ((dir = readdir(d)) != NULL)
    {
      printf("%s\n", dir->d_name);
    }

    closedir(d);
  }

  return(0);
}

Any clue on what the problem might be? I'm thinking, at this point, that we may need to wrap a C function to get the proper list.

By the way, the files in the directory all have the same permissions, none are symbolic or hard links, everything looks normal from the ls -l command on a terminal window.

Keith Hogan
  • 117
  • 1
  • 9
  • Do your programs return an error? Have you tried the same method as in C, where you make multiple calls to `Readdir`? – JimB Oct 12 '17 at 22:36
  • No errors are returned. The golang ReadDir returns a slice of file information, so, you iterate through that. We're still looking at other golang ways of accomplishing this. – Keith Hogan Oct 12 '17 at 23:10
  • One other important piece of information is that this is running on CentOS 7.4. – Keith Hogan Oct 12 '17 at 23:10
  • No, I mean using `Readdir` directly with an argument >0, not `ioutil.ReadDir`. – JimB Oct 12 '17 at 23:13
  • So, you mean os.Readdir, and use a number like 20, and loop until all the way through the directory, twenty at a time. We will try that. Thanks. – Keith Hogan Oct 12 '17 at 23:24
  • 1
    I had the precise problem you describe, and have filed an issue: https://github.com/golang/go/issues/24015. As a workaround, you can force the CIFS mount to use version 1 ("vers=1.0"), though that's far from ideal. – Nerdmaster Feb 21 '18 at 21:06
  • Ignore my prior comment - the CIFS version only helped in some cases, not all. – Nerdmaster Feb 27 '18 at 21:41

0 Answers0