Qt vs Boost filesystem recursive file count

Question

void countFiles() {   
  QString root_path("C:\\");

  QTime timer;
  timer.start();

  std::uint64_t count = 0;
  std::queue<QString> qt_dirs;
  qt_dirs.push(root_path);

  while (!qt_dirs.empty()) {
    auto dir_path = qt_dirs.front();
    qt_dirs.pop();

    QDir dir(dir_path);

    count += dir.entryList(QDir::Dirs | QDir::Files | QDir::NoSymLinks | QDir::NoDotAndDotDot).size();

    for (auto &sub_dir_path : dir.entryList(QDir::Dirs | QDir::NoSymLinks | QDir::NoDotAndDotDot)) {
      qt_dirs.push(dir.filePath(sub_dir_path));
    }
  }

  qDebug() << Q_FUNC_INFO << "found" << count << "entries, and it took" << timer.elapsed() << "ms";

  timer.start();
  count = 0;

  std::queue<boost::filesystem::path> dirs;
  dirs.push(root_path.toStdString());

  while (!dirs.empty()) {
    auto dir_path = dirs.front();
    dirs.pop();

    try {
      auto iterator_range = boost::make_iterator_range(boost::filesystem::directory_iterator(dir_path), {});

      for (auto &entry : iterator_range) {

        auto entry_status = entry.status();

        if (boost::filesystem::is_symlink(entry_status)) continue;
        if (boost::filesystem::is_directory(entry_status)) dirs.push(entry.path());

        ++count;
      }
    } catch(boost::filesystem::filesystem_error &fe) {
      continue;
    }
  }

  qDebug() << Q_FUNC_INFO << "found" << count << "entries, and it took" << timer.elapsed() << "ms";
}

Can someone explain to me, or at least give me a hint, why these 2 blocks return completely different counts of files? They both should only count directories and files, skip any symlinks. But still, on Windows this differs by around 20%

void VolumeFileTreeModel::countFiles() found 502780 entries, and it took 97549 ms
void VolumeFileTreeModel::countFiles() found 622208 entries, and it took 17022 ms

Add some print statements to print out the files and directories that are being counted, and diff the output. — RichieHindle, Apr 30 '15 at 14:08
Could the person that downvoted, also explain what is wrong with the question? .. And thank you, I will try to diff the outputs — Pinker, Apr 30 '15 at 16:55

Rudolfs Bundulis · Answer 1 · 2015-04-30T15:40:44.990

0

One notable difference is the QDir::NoDotAndDotDot filter. Can you try and modify the Boost version to skip these directories as well (I guess comparing the directory name to .. and . should be fine? Since now I guess they will be counted in by Boost and not by Qt.

Update

I'll have a second try - how about adding Qt filters QDir::Hidden and QDir::System? As I understand from [this] question at least hidden files are included in Boost.

edited Apr 30 '15 at 15:40

answered Apr 30 '15 at 14:20

Rudolfs Bundulis

11,636
6
33
71

The boost documentation at http://www.boost.org/doc/libs/1_46_1/libs/filesystem/v3/doc/reference.html#Class-directory_iterator says *"Directory iteration shall not yield directory entries for the current (dot) and parent (dot dot) directories."* (And the code would surely have never terminated if it did.) – RichieHindle Apr 30 '15 at 15:19
Yeah I thought that it should trigger an infinite recursion:) I will remove the answer since it's useless. – Rudolfs Bundulis Apr 30 '15 at 15:33
@RichieHindle since the question makes me curious I'll try one more suggestion:D – Rudolfs Bundulis Apr 30 '15 at 15:41

Qt vs Boost filesystem recursive file count

1 Answers1