Can I use a mask to iterate files in a directory with Boost?

Question

I want to iterate over all files in a directory matching something like somefiles*.txt.

Does boost::filesystem have something built in to do that, or do I need a regex or something against each leaf()?

score 76 · Accepted Answer · edited Apr 22 '19 at 22:30

76

EDIT: As noted in the comments, the code below is valid for versions of boost::filesystem prior to v3. For v3, refer to the suggestions in the comments.

boost::filesystem does not have wildcard search, you have to filter files yourself.

This is a code sample extracting the content of a directory with a boost::filesystem's directory_iterator and filtering it with boost::regex:

const std::string target_path( "/my/directory/" );
const boost::regex my_filter( "somefiles.*\.txt" );

std::vector< std::string > all_matching_files;

boost::filesystem::directory_iterator end_itr; // Default ctor yields past-the-end
for( boost::filesystem::directory_iterator i( target_path ); i != end_itr; ++i )
{
    // Skip if not a file
    if( !boost::filesystem::is_regular_file( i->status() ) ) continue;

    boost::smatch what;

    // Skip if no match for V2:
    if( !boost::regex_match( i->leaf(), what, my_filter ) ) continue;
    // For V3:
    //if( !boost::regex_match( i->path().filename().string(), what, my_filter ) ) continue;

    // File matches, store it
    all_matching_files.push_back( i->leaf() );
}

(If you are looking for a ready-to-use class with builtin directory filtering, have a look at Qt's QDir.)

edited Apr 22 '19 at 22:30

Gidy

426
6
16

answered Aug 11 '09 at 09:06

Julien-L

5,267
3
34
51

21

thanks for the very complete answer. two notes for others: 1) leaf is deprecated in filesystem v3 (current default), use path().filename() instead 2) if the filter criterion is the extension (very common) it is easier to use i->path().extension() == ".txt" [for example] than regex – alfC Nov 04 '11 at 07:01
18

leaf() is now deprecated. i->leaf() can be replaced by i->path().string() or i->path().filename().string() if you just want the filenames – Fuzz Jun 27 '12 at 01:33
6

The backslash in regex has to be escaped `"somefiles.*\\.txt"` – David L. Apr 26 '14 at 11:37
@Julien-L I actually had to change the filter to `const boost::regex my_filter( tarrget_path + "somefiles.*\.txt" );` – David Doria Dec 17 '15 at 13:59

score 9 · Answer 2 · answered May 18 '15 at 14:05

There is a Boost Range Adaptors way:

#define BOOST_RANGE_ENABLE_CONCEPT_ASSERT 0
#include <boost/filesystem.hpp>
#include <boost/range/adaptors.hpp>

namespace bfs = boost::filesystem;
namespace ba = boost::adaptors;

const std::string target_path( "/my/directory/" );
const boost::regex my_filter( "somefiles.*\.txt" );
boost::smatch what;

for (auto &entry: boost::make_iterator_range(bfs::directory_iterator(target_path), {})
| ba::filtered(static_cast<bool (*)(const bfs::path &)>(&bfs::is_regular_file))
| ba::filtered([&](const bfs::path &path){ return boost::regex_match(path.filename().string(), what, my_filter); })
)
{
  // There are only files matching defined pattern "somefiles*.txt".
  std::cout << entry.path().filename() << std::endl;
}

For those who run into this nowadays: std::regex_match as well as boost::regex_match do no longer accept temporary string. For the given example above this means that the lambda body needs to be adjusted to something like auto oFile=path,filename().string(); return xxx::regex_match(oFile, ...). — gilgamash, Aug 11 '16 at 08:53

David L. · Answer 3 · 2014-12-16T10:47:09.847

My solution is essentially the same as Julien-L, but encapsulated in the include file it is nicer to use. Implemented using boost::filesystem v3. I guess that something like that is not included in the boost::filesystem directly because it would introduce dependency on boost::regex.

#include "FilteredDirectoryIterator.h"
std::vector< std::string > all_matching_files;
std::for_each(
        FilteredDirectoryIterator("/my/directory","somefiles.*\.txt"),
        FilteredDirectoryIterator(),
        [&all_matching_files](const FilteredDirectoryIterator::value_type &dirEntry){
                all_matching_files.push_back(dirEntry.path());
            }
        );

alternatively use FilteredRecursiveDirectoryIterator for recursive sub directories search:

#include "FilteredDirectoryIterator.h"
std::vector< std::string > all_matching_files;
std::for_each(
        FilteredRecursiveDirectoryIterator("/my/directory","somefiles.*\.txt"),
        FilteredRecursiveDirectoryIterator(),
        [&all_matching_files](const FilteredRecursiveDirectoryIterator::value_type &dirEntry){
                all_matching_files.push_back(dirEntry.path());
            }
        );

FilteredDirectoryIterator.h

#ifndef TOOLS_BOOST_FILESYSTEM_FILTEREDDIRECTORYITERATOR_H_
#define TOOLS_BOOST_FILESYSTEM_FILTEREDDIRECTORYITERATOR_H_

#include "boost/filesystem.hpp"
#include "boost/regex.hpp"
#include <functional>

template <class NonFilteredIterator = boost::filesystem::directory_iterator>
class FilteredDirectoryIteratorTmpl
:   public std::iterator<
    std::input_iterator_tag, typename NonFilteredIterator::value_type
    >
{
private:
    typedef std::string string;
    typedef boost::filesystem::path path;
    typedef
        std::function<
            bool(const typename NonFilteredIterator::value_type &dirEntry)
            >
        FilterFunction;

    NonFilteredIterator it;

    NonFilteredIterator end;

    const FilterFunction filter;

public:

    FilteredDirectoryIteratorTmpl();

    FilteredDirectoryIteratorTmpl(
        const path &iteratedDir, const string &regexMask
        );

    FilteredDirectoryIteratorTmpl(
        const path &iteratedDir, const boost::regex &mask
        );

    FilteredDirectoryIteratorTmpl(
        const path &iteratedDir,
        const FilterFunction &filter
        );

    //preincrement
    FilteredDirectoryIteratorTmpl<NonFilteredIterator>& operator++() {
        for(++it;it!=end && !filter(*it);++it);
        return *this;
    };

    //postincrement
    FilteredDirectoryIteratorTmpl<NonFilteredIterator> operator++(int) {
        for(++it;it!=end && !filter(*it);++it);
        return FilteredDirectoryIteratorTmpl<NonFilteredIterator>(it,filter);
    };
    const boost::filesystem::directory_entry &operator*() {return *it;};
    bool operator!=(const FilteredDirectoryIteratorTmpl<NonFilteredIterator>& other)
    {
        return it!=other.it;
    };
    bool operator==(const FilteredDirectoryIteratorTmpl<NonFilteredIterator>& other)
    {
        return it==other.it;
    };
};

typedef
    FilteredDirectoryIteratorTmpl<boost::filesystem::directory_iterator>
    FilteredDirectoryIterator;

typedef
    FilteredDirectoryIteratorTmpl<boost::filesystem::recursive_directory_iterator>
    FilteredRecursiveDirectoryIterator;

template <class NonFilteredIterator>
FilteredDirectoryIteratorTmpl<NonFilteredIterator>::FilteredDirectoryIteratorTmpl()
:   it(),
    filter(
        [](const boost::filesystem::directory_entry& /*dirEntry*/){return true;}
        )
{

}

template <class NonFilteredIterator>
FilteredDirectoryIteratorTmpl<NonFilteredIterator>::FilteredDirectoryIteratorTmpl(
    const path &iteratedDir,const string &regexMask
    )
:   FilteredDirectoryIteratorTmpl(iteratedDir, boost::regex(regexMask))
{
}

template <class NonFilteredIterator>
FilteredDirectoryIteratorTmpl<NonFilteredIterator>::FilteredDirectoryIteratorTmpl(
    const path &iteratedDir,const boost::regex &regexMask
    )
:   it(NonFilteredIterator(iteratedDir)),
    filter(
        [regexMask](const boost::filesystem::directory_entry& dirEntry){
            using std::endl;
            // return false to skip dirEntry if no match
            const string filename = dirEntry.path().filename().native();
            return boost::regex_match(filename, regexMask);
        }
        )
{
    if (it!=end && !filter(*it)) ++(*this);
}

template <class NonFilteredIterator>
FilteredDirectoryIteratorTmpl<NonFilteredIterator>::FilteredDirectoryIteratorTmpl(
    const path &iteratedDir, const FilterFunction &filter
    )
:   it(NonFilteredIterator(iteratedDir)),
    filter(filter)
{
    if (it!=end && !filter(*it)) ++(*this);
}

#endif

score 1 · Answer 4 · answered Aug 11 '09 at 00:41

1

I believe the directory_iterators will only provide all files in a directory. It up to you to filter them as necessary.

answered Aug 11 '09 at 00:41

jon hanson

8,722
2
37
61

score 1 · Answer 5 · answered Dec 09 '15 at 21:39

The accepted answer did not compile for me even when I used i->path().extension() instead of leaf(). What did work for me was an example from this website. Here's the code, modified, to apply a filter:

vector<string> results;
filesystem::path filepath(fullpath_to_file);
filesystem::directory_iterator it(filepath);
filesystem::directory_iterator end;
const boost::regex filter("myfilter(capturing group)");
BOOST_FOREACH(filesystem::path const &p, make_pair(it, end))
{
     if(is_regular_File(p))
     {
          match_results<string::const_iterator> what;
          if (regex_search(it->path().filename().string(), what, pidFileFilter, match_default))
          {
               string res = what[1];
               results.push_back(res);
          }
     }
}

I'm using Boost version: 1.53.0.

Why we don't all just use glob() and some regex is beyond me.

score 0 · Answer 6 · edited May 23 '17 at 11:54

0

As mentioned at the end of Julien-L's post QDir is exactly what you want.
https://qt-project.org/doc/qt-5.0/qtcore/qdir.html#QDir-3

edited May 23 '17 at 11:54

Community

1
1

answered Mar 18 '14 at 15:08

Jonathan Mee

37,899
23
129
288

score 0 · Answer 7 · edited Jan 07 '20 at 12:40

I was looking for a solution to this earlier and I think my solution is the simplest

#include <boost/filesystem.hpp>
#include <boost/regex.hpp>
#include <boost/iterator/iterator_facade.hpp>
#include <boost/exception/all.hpp>

struct dir_filter_iter
        : public boost::iterator_facade<
                dir_filter_iter,
                boost::filesystem::path,
                boost::forward_traversal_tag,
                boost::filesystem::path
        >
{
        using path = boost::filesystem::path;
        using impl_type = boost::filesystem::directory_iterator;

        dir_filter_iter():impl_(){}
        dir_filter_iter(path p, boost::regex rgx):impl_(std::move(p)),rgx_(std::move(rgx)){
                namespace bf = boost::filesystem;
                if( ! bf::is_directory(p) ){
                        BOOST_THROW_EXCEPTION(
                                boost::enable_error_info(std::domain_error("not a dir"))
                                << boost::errinfo_file_name(p.string()));
                }

             //advance to first matching item if impl_ is not already at end()
             if(impl_ != impl_type()) {
                  std::string s(impl_->path().string());
                  if( !boost::regex_match( s, rgx_ )) increment();
             }
        }
        private:
        friend class boost::iterator_core_access;
        bool equal(const dir_filter_iter& that)const{
                return this->impl_ == that.impl_;
        }
        void increment(){
                assert( impl_ != impl_type() );
                for(;;){
                        ++impl_;
                        if( impl_ == impl_type() )
                                break;
                        std::string s(impl_->path().string());
                        if( boost::regex_match( s, rgx_ ) ){
                                break;
                        }
                }
        }
        path dereference()const{
                assert( impl_ != impl_type() );
                return *impl_;
        }
        impl_type impl_;
        boost::regex rgx_;
};
struct dir_filter_iter_maker{
        using value_type = dir_filter_iter;

        explicit dir_filter_iter_maker(boost::regex rgx):rgx_(rgx){}

        value_type make()const{
                return value_type();
        }
        value_type make(boost::filesystem::path p)const{
                return value_type(std::move(p),rgx_);
        }
        template<typename... Args>
        auto operator()(Args&&... args)->decltype(make(args...)){
                return this->make(std::forward<Args>(args)...);
        }
        private:
        boost::regex rgx_;
};

Then you can do

    dir_filter_iter_maker di_maker(boost::regex(R"_(.*\.hpp)_"));
    std::for_each( di_maker(p), di_maker(), [](const bf::path& p){std::cout << p.string() << "\n";});

Can I use a mask to iterate files in a directory with Boost?

7 Answers7

Linked

Related