boost::filesystem recursively getting size of each file

Question

Why does this code throw an error when the argument is a directory?

Using boost::recursive_directory_iterator and using std::cout statements, I can see that it never prints a directory; only files. But, when I try to call boost::filesystem::file_size(), an error is thrown basically saying I'm trying to get the file size of a directory.

Error (argument was "/home"):

terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
  what():  boost::filesystem::file_size: Operation not permitted: "/home/lost+found"
Aborted

#include <iostream>
#include <boost/filesystem.hpp>

namespace fs = boost::filesystem;

int main(int argc, char* argv[])
{
    if (argc != 2) return -1;

    const fs::path file{argv[1]};

    if (!fs::exists(file)) return -1;

    if (fs::is_regular_file(file))
        std::cout << file << "   [ " << fs::file_size(file) << " ]\n";

    else if (fs::is_directory(file))
        for (const fs::directory_entry& f : fs::recursive_directory_iterator{file})
            std::cout << f.path().filename() << "   [ " << fs::file_size(f.path()) << " ]\n";
}

Compiled with: g++ -Wall -Wextra -pedantic-errors -std=c++14 -lboost_system -lboost_filesystem -O2 -Os -s test3.cpp -o test3

*"Using boost::recursive_directory_iterator and using std::cout statements, I can see that it never prints a directory; only files."* -- Are you sure about that? It should print directories too. It does when I use it. — Benjamin Lindley, Jul 18 '17 at 14:42
@BenjaminLindley I guess it just throws an exception on the first directory met and program terminates. So OP doesn't see any directories. — Edgar Rokjān, Jul 18 '17 at 14:44
@EdgarRokyan: Could be. I assumed that he meant he tried it with only printing the filename, and not trying to get the file size. Hence the *"But, when I try to call boost::filesystem::file_size() ..."*. — Benjamin Lindley, Jul 18 '17 at 14:47
When I tested `boost::filesystem::recursive_directory_iterator`, the program printed out a large amount of text. I missed that it prints out directories as well as files. Thanks @BenjaminLindley. — cppxor2arr, Jul 20 '17 at 06:49

score 4 · Accepted Answer · edited Jun 30 '20 at 11:51

The error you get:

terminate called after throwing an instance of 'boost::filesystem::filesystem_error' what(): boost::filesystem::file_size: Operation not permitted: "/home/lost+found" Aborted

It means that it can't get the size of /home/lost+found. Normally, lost+found is a folder and file_size only get the size of regular files.

I understand that the loop does not show the name of this folder. It may be because the compiler is evaluating fs::file_size(f.path()) and throwing the exception before calling operator<< for the file name so it does not get printed.

I think the loop should be modified to check for a regular file before asking for the size:

for (const fs::directory_entry& f : fs::recursive_directory_iterator(folder)) {
  if (fs::is_regular_file(f.path())) {
    std::cout << f.path().filename() << "   [ " << fs::file_size(f.path()) << " ]\n";
  }
}

sehe · Answer 2 · 2017-07-18T15:13:02.240

Try to actually recursively get the size:

size_t du(fs::path p) {
    return fs::is_regular_file(p)
       ? file_size(p)
       : boost::accumulate(fs::directory_iterator{p}, 0ull, [](auto a, auto p){return a+du(p);});
}

This will work for directories by summing (accumulate) the files in all underlying directories.

Live On Coliru

#include <iostream>
#include <boost/filesystem.hpp>
#include <boost/range/numeric.hpp>

namespace fs = boost::filesystem;
size_t du(fs::path p) {
    std::cout << __FUNCTION__ << "(" << p << ")\n";
    return fs::is_regular_file(p)
       ? file_size(p)
       : boost::accumulate(fs::directory_iterator{p}, 0ull, [](auto a, auto p){return a+du(p);});
}

int main(int argc, char* argv[])
{
    if (argc != 2) return -1;

    std::cout << "Size is " << du(argv[1]) << "\n";
}

With the debug std::cout enabled:

Size is du(".")
du("./main.cpp")
du("./a.out")
22435

Won't this get the size of files nested in directories multiple times? `recursive_directory_iterator` already handles the recursion of going down into nested directories, but then you're handling it again by calling `du(p)` recursively. — Benjamin Lindley, Jul 18 '17 at 15:10
Sheepish grin. @BenjaminLindley Fixed! I guess you could use `recursive_`* and just feed it into `accumulate — sehe, Jul 18 '17 at 15:13

boost::filesystem recursively getting size of each file

2 Answers2