C++17 custom iterator in a pimpl idiom context

Question

I want to know how to have a c++ class to be iterable (stl compatible) without exposing the implementation ?

The structure of the project is like :

Stream

class Stream
{
public:
    Stream();

    [...]

    StreamIterator iter()
    {
        return StreamIterator(this);
    }

private:
    class impl;
    std::unique_ptr<impl> pimpl;
};

StreamFilter

class StreamFilter
{
public:
    StreamFilter();

    [...]

private:
    class impl;
    std::shared_ptr<impl> pimpl;
};

StreamIterator

class StreamIterator
{
public:
    StreamIterator(Stream* streamToFilter);

    [...]

    void addFilter(StreamFilter* filter);
    void removeFilter(StreamFilter* filter);

    [...]

private:
    class impl;
    std::unique_ptr<impl> pimpl;
};

StreamFilter is a base class for differents filtering strategies.

For simplicity in the sample code, I used raw memory pointers, of course in a real exploitation code I use intelligents pointers : shared, weak ...

I want to allow the StreamIterator become iterable in a STL way, doing :

StreamIterator iter = stream.iter();
iter.addFilter(new FilterByOffset([...with parameters...]));
for (auto item : iter)
{
    [...doing something with filtered items...]
}

The basic way is to add some accessors to allow range-based for loop.

StreamIterator

class StreamIterator
{
public:
    StreamIterator(Stream* streamToFilter);

    [...]

    iterator begin();
    iterator end();

    const_iterator cbegin() const;
    const_iterator cend() const;

    [...]

    void addFilter(StreamFilter* filter);
    void removeFilter(StreamFilter* filter);

    [...]

private:
    class impl;
    std::unique_ptr<impl> pimpl;
};

Where iterator and const_iterator are basically typedef's to the internal container iterators. And this is the problem.

First, I don't want to expose private implementation in the StreamIterator header. And so, iterator and const_iterator are not allowed here.

Second, because of the stream filtering strategy, the iterators returned are not just alias to some internal stl containers. In the internal implementation, I need to call the filters in a functor way to check if the item need to be exclude or not.

The only type allowed in the StreamIterator header is the type of the item object returned.

Is there a way to do that?

Thank you very much!

Additional Information:

Maybe this declaration is a way to allow a private implementation, I need to investigate more :

StreamIterator

class StreamIterator
{
public:
    StreamIterator(Stream* streamToFilter);

    [...]

    struct iterator
    {
        Object::Ptr operator*();
        iterator& operator++();
        bool operator!= (const iterator& it) const;
    };

    typedef typename StreamIterator::iterator iterator;

    iterator begin();
    iterator end();

    [...]

    void addFilter(StreamFilter* filter);
    void removeFilter(StreamFilter* filter);

    [...]

private:
    class impl;
    std::unique_ptr<impl> pimpl;
};

Answer 1

First, don't call it StreamIterator; an iterator is a pointer-like object for which 2 of them can specify a range. Your StreamIterator doesn't have this. Nothing good can come of reusing the well defined iterator term here.

---

Now, your StreamIterator is some kind of range of iterators. So we'll call it a StreamRange.

In order to hide how StreamRange can be iterated over, you have to hide how the iterators it uses work.

And this -- hiding the implementation detalis of a stream iterator -- has a substantial cost to it.

When iterating over a loop, each step in the loop involves ++ and * and an ==. Throwing a pointer indirection and a vtable lookup (or equivalent) on each of those will make your loops much slower.

But here is how to do it.

template<class Value>
struct any_input_iterator {
  using difference_type = std::ptrdiff_t;
  using value_type=Value;
  using pointer = value_type*;
  using reference = value_type;
  using iterator_category = std::input_iterator_tag;

private:
  struct vtable_t {
    bool(*equal)( std::any const& lhs, std::any const& rhs ) = 0;
    Value(*get)( std::any& ) = 0;
    void(*inc)( std::any& ) = 0;
  };
  vtable_t const* vtable = 0;
  std::any state;

  template<class U>
  static vtable_t make_vtable() {
    return {
      [](std::any const& lhs, std::any const& rhs)->bool {
        return std::any_cast<U const&>(lhs) == std::any_cast<U const&>(rhs);
      },
      [](std::any& src)->Value {
        return *std::any_cast<U&>(src);
      },
      [](std::any& src) {
        ++std::any_cast<U&>(src);
      }
    };
  }
  template<class U>
  static vtable_t const* get_vtable() {
    static const auto vtable = make_vtable<U>();
    return &vtable;
  }
public:
  template<class U,
    std::enable_if_t<!std::is_same<std::decay_t<U>, any_input_iterator>{}, bool> = true
  >
  any_input_iterator(U&& u):
    vtable(get_vtable<std::decay_t<U>>()),
    state(std::forward<U>(u))
  {}

  any_input_iterator& operator++() { vtable->inc(state); return *this; }
  any_input_iterator operator++(int) { auto tmp = *this; ++*this; return tmp; }
  reference operator*() { return vtable->get(state); }
  friend bool operator==( any_input_iterator const& lhs, any_input_iterator const& rhs ) {
    if (lhs.vtable != rhs.vtable) return false;
    if (!lhs.vtable) return true;
    return lhs.vtable->equal( lhs.state, rhs.state );
  }
  friend bool operator!=( any_input_iterator const& lhs, any_input_iterator const& rhs ) {
    return !(lhs==rhs);
  }

  struct fake_ptr {
    Value t;
    Value* operator->()&&{ return std::addressof(t); }
  };

  fake_ptr operator->()const { return {**this}; }
};

there are probably some typoes, but this is basic type erasure. Boost does a better job at this.

I only supported input iterators. If you want to support forward iterators, you have to change up some typedefs and return references and the like.

any_input_iterator<int> begin();
any_input_iterator<int> end();

that is, however, enough to let someone iterate over your range in question.

It will be slow, but it will work.