Standard-conforming transform_iterator

Question

My goal would be to have an iterator that iterates over elements of type T, but in my case T is not really usable for my end-users. Instead, end-users should use a wrapper W that has a much more useful interface.

In order to construct a W, we need a T plus a reference or pointer to an additional data structure.

The problem is that I will never store elements as W. Instead, elements are always stored as T and only wrapped on-demand. Therefore, my users have to iterator over a data structure holding Ts.

My idea was to write a custom iterator for these data structures that itself iterates over the stored Ts, but upon dereferencing will return W instead. I started looking into how this can be implemented and found various information on this topic, including how to deal with the iterator's reference typedef not actually being a reference. This includes the arrow_proxy trick for implementing operator-> in such cases.

However, I have also attempted to read in the standard to see what it has to say about such iterators. My resource here is this and there it clearly states that as soon as we are dealing with forward iterators, reference is expected to be a (const) reference to the value_type. This is supported by this question.

This makes me wonder whether it is even possible to reasonably implement such a transform_iterator that remains standard conforming if one intends to be using it as a forward_iterator or above?

One way that I could come up with would be to declare the value_type of my iterator to W and then keep a member variable of type W around such that operator* could be implemented like this:

class transform_iterator {
    value_type = W;
    reference = W &;
    // ...

    reference operator*() const {
        m_wrapper = W(< obtain current T >, m_struct);
        return m_wrapper;
    }

    mutable W m_wrapper;
    SeparateDataStructure m_truct;
};

However, this approach seems rather hacky to me. On top of that would this increase the iterators size seemingly considerably, which might or might not be an issue (in the long run).

Note 1: I know that Boost.iterator provides a transform_iterator, but I can't quite follow through the implementation of what iterator category they actually apply to these types of iterators. However, it does seem like they base the category on the result type of the supplied function (in some way), which would suggest that at least in their implementation it is possible for the category to be different from input_iterator_tag (though maybe the only other option is output_iterator_tag?).

Note 2: The question linked above also suggest the same workaround that I sketched here. Does that indicate that there is no better way?

TL;DR: Is there a better way to achieve e.g. a forward iterator that transforms the iterated type on dereference to a different type than to store a member of that type in the iterator itself and update that member on every dereference?

You don't say why your iterator needs to be a forward iterator specifically, instead of just an input iterator. The reasons why _ForwardIterator_ must yield a reference don't apply to _InputIterator_, so ... do you actually need multiple passes? — Useless, Jan 13 '23 at 16:26
I think cheating *is* the normal way. You make a "ForwardIterator-but" — user253751, Jan 13 '23 at 16:26
@Useless no I don't need multiple passes but I want `operator->` support. But I would like to avoid extensions to the standard interface in order to be able to potentially switch out the iterator implementation for another of that same category without breaking client code — Raven, Jan 13 '23 at 16:42
I'd read the standard rather in the form of *'`(void)*a, *a` is equivalent to `*a`'* – I conclude from that if you modify a dereferenced iterator then these modifications should be visible if you dereference the iterator *again*. As long as your wrapper can satisfy this condition (i.e. modifying `W` results in only modifying the underlying `T`), then it should be fine just to return a `W` by value from both `operator*` *and* `operator->`... — Aconcagua, Jan 13 '23 at 16:46
Actually, now when re-reading the docs for `Boost.iterator`, I understand what they complain about with regards to iterator categories and what [this paper](https://www.boost.org/doc/libs/1_81_0/libs/iterator/doc/new-iter-concepts.pdf) is about. I guess to get what I want I'll have to opt into custom iterator tags as well... — Raven, Jan 13 '23 at 16:47
[_InputIterator_](https://en.cppreference.com/w/cpp/named_req/InputIterator) already has `operator->` support. If you don't need multi-pass, you don't need _ForwardIterator_ and all your problems go away. Can you explain why you actually need some other iterator category? — Useless, Jan 13 '23 at 16:49
@Aconcagua good point. This is also in agreement with what I have read in different places. The only place that I can imagine where this breaks would be in implicit conversions. So if `T &` is required to implicitly convert to another type, this conversion won't happen if you return a wrapper that itself is implicitly convertible to `T &` as that'd then be two chained implicit conversions, which is not allowed (afaik) — Raven, Jan 13 '23 at 16:49
@Useless oh yes indeed. How could I miss that? I was staring at this all day long and I even implemented it in this way, yet I was obsessed with my iterator being a forward iterator to have `operator->` support... *facepalm* Thanks for pointing this out! — Raven, Jan 13 '23 at 16:51
@Raven So then why speaking of [conversion *sequences*](https://en.cppreference.com/w/cpp/language/implicit_conversion)? I'm not aware of this being true, though such conversions are *not* applied on templates – here we indeed can get a bit of trouble, but that's all alike, too, with returning references to `W` (i.e. `template void f(U const& u); f(*iterator)` will instantiate `f` with `W`, not `T`, no matter if returning by reference or by value). — Aconcagua, Jan 13 '23 at 17:01
@Aconcagua yeah. actually, the docs you linked seem to suggest that any user-defined conversion may again be followed by a "regular" implicit conversion, nullifying my argument from above. As far as the templates are concerned, in my case the template being instatiated on `W` rather than `T` would (probably) be what I want. But in the general case (which this question is all about) this would indeed be a problem. — Raven, Jan 13 '23 at 17:13

score 1 · Accepted Answer · answered Jan 13 '23 at 17:57

First, choose the most restrictive iterator category you can get away with:

if you don't need to allow multiple passes, use InputIterator and just return a temporary W by value

This still has all the usual iterator methods (operator*, operator->, both operator++ etc.)
if you do need multiple passes, you need a ForwardIterator with its additional requirement to return an actual reference.

As you say, this can only be done by storing a W somewhere (in the iterator or off to the side).

The big problem is with mutable forward iterators: mutating *i must also affect *j if i == j, and that means your W can't be a standalone value type, but must be some kind of write-through proxy. That's not impossible, but you can't simply take some existing type and use it like this.

If you have C++20 access, you could probably save some effort by just using a transform_view - although this may be outweighed by the work of changing everything else to use ranges rather than raw iterators.

Standard-conforming transform_iterator

1 Answers1