13

I have a templated bidirectional iterator. I do not want to make it random access because the it += n operation would not be constant time. However, the it2 - it1 operation is constant time. I wanted to specialize std::distance() for this iterator so that algorithms that use it (such as std::vector::assign()) can make use of the efficient difference operation. How can I do this if the iterator is a template?

Here's a toy example:

#include <iterator>
#include <iostream>

// template bidirectional iterator
template<typename T>
class iter : public std::iterator<std::bidirectional_iterator_tag, T> {
    T *ptr;
public:
    iter(T *ptr) : ptr(ptr) { }

    iter() = default;
    iter(const iter &) = default;
    iter &operator = (const iter &) = default;

    T *operator * () { return ptr; }

    bool operator == (const iter &it) { return ptr == it.ptr; }
    bool operator != (const iter &it) { return ptr != it.ptr; }

    iter &operator ++ () { ++ptr; return *this; }
    iter operator ++ (int) { iter tmp(*this); operator++(); return tmp; }

    iter &operator -- () { --ptr; return *this; }
    iter operator -- (int) { iter tmp(*this); operator--(); return tmp; }

    // Would not be used for a bidirectional iterator.
    // Implemented only so we can use it in std::distance() below.
    ptrdiff_t operator - (const iter &it) { return ptr - it.ptr; }
};

namespace std {
    // We could specialize std::distance() for iter<int> like this:
    template<>
    iter<int>::difference_type distance(iter<int> first, iter<int> last) {
        std::cout << "my distance called\n";
        return last - first;
    }

    // QUESTION: Can we do it in general, for iter<T> ?
}

// Just to test that everything works as intended.
int main() {
    int arr[5];
    iter<int> it1(&arr[0]);
    iter<int> it2(&arr[5]);

    std::cout << std::distance(it1, it2) << std::endl;

    return 0;
}

This is a follow-up of Is it reasonable to overload std functions such as std::distance?

We could in principle do something like this:

namespace std {
    template<class T>
    typename iter<T>::difference_type distance(iter<T> first, iter<T> last) {
        std::cout << "my distance called\n";
        return last - first;
    }
}

But that would be an overload of std::distance(), which is not allowed for std namespace functions according to the standard.

Szabolcs
  • 24,728
  • 9
  • 85
  • 174
  • 1
    Ah, can't partially specialize a template member function. And can't overload it, like you previously asked. Tough spot. – StoryTeller - Unslander Monica Nov 06 '17 at 10:08
  • @StoryTeller Since this is just an optimization opportunity, perhaps I could only provide specializations for the few types that `iter` is most commonly used with. – Szabolcs Nov 06 '17 at 10:15
  • if you know the distance upfront, can't you just vector::reserve before vector::assign ? – Massimiliano Janes Nov 06 '17 at 10:16
  • Yes, that could work. It's a shame the customization points aren't more numerous. This is a good example of when adding a definition to namespace std may not be so unreasonable. – StoryTeller - Unslander Monica Nov 06 '17 at 10:17
  • 2
    Why not make your `distance` function in the same namespace as your templated iterator and rely on ADL with unqualified calls to `distance`? – Rerito Nov 06 '17 at 10:17
  • @MassimilianoJanes I could, but this would be used by people other than myself. If there is no good solution, I can accept that as an answer. It's not critical to implement this optimization. I was simply trying to understand if it is possible at all (in a standards compliant manner). – Szabolcs Nov 06 '17 at 10:18
  • It's actually an interesting point made by Rerito. I wonder how implementations handle custom iterator types. – StoryTeller - Unslander Monica Nov 06 '17 at 10:19
  • ok, but as far as I can tell, there's no guarantee that distance() will be used anyway in vector::assign and such ( be it in std or via ADL) .... – Massimiliano Janes Nov 06 '17 at 10:19
  • 2
    @MassimilianoJanes - You mean a standard algorithm will recreate the logic of `std::distance` instead of using it? Sure, it is allowed to do so by the dry letter of the standard, but really? – StoryTeller - Unslander Monica Nov 06 '17 at 10:20
  • 5
    @Szabolcs: It's still a good idea. You may be a bit pessimistic about "not used by other people than yourself". The problem you're facing (no overloading in `std::`) is well-known, and the standard solution is to use `using std::foo; foo(arg1, arg2)`. ADL preferred, with fallback to `std::foo`. – MSalters Nov 06 '17 at 10:22
  • @storyteller no, I mean that there's nothing in vector::assign guaranteeing that it will call distance() internally, or am I missing something ? – Massimiliano Janes Nov 06 '17 at 10:22
  • @MassimilianoJanes - Of course there isn't. The point behind the OP is that if it does (some implementations may call it), it's quite likely to use `std::distance` (not so much to rely on ADL IMO, but you never know). – StoryTeller - Unslander Monica Nov 06 '17 at 10:23
  • @storyteller, do you mean that an implementation of vector::assign will internally call distance() when given a *bidirectional iterator* ? it's legal, but it's not obvious to me that's a sufficiently plausible assumption motivating such an optimization ... – Massimiliano Janes Nov 06 '17 at 10:28
  • @MassimilianoJanes - Stranger things have happened. It may like to call it once and cache the result. An implementation may like to pre-allocate sufficient space. `std::distance` has to come up there for that. – StoryTeller - Unslander Monica Nov 06 '17 at 10:30
  • @storyteller ok, you're right, I checked clang stdlib, and indeed it does reserve via distance for forward iterators up. That said, the call is qualified. – Massimiliano Janes Nov 06 '17 at 10:33
  • @MSalters Thanks for pointing that out. I did not have a good understanding of ADL but now I looked it up ([it works](http://coliru.stacked-crooked.com/a/1d25b0150c93cfec)). This seems to basically require C++ programmers to know which `std` namespace functions are reasonable candidates for things like this, and call them in the manner you described. I wonder if there are any other functions than [swap()](http://en.cppreference.com/w/cpp/concept/Swappable) which are typically used this way. – Szabolcs Nov 06 '17 at 10:41
  • @Szabolcs: This is mostly a concern for template library writers (i.e. Boost), and they are aware of the issue. For ordinary libraries it's rarely needed (they know up front which types they're handling). – MSalters Nov 06 '17 at 10:48
  • 2
    Per [global.functions]/4, the standard library doesn't make ADL-enabled calls to non-operator functions unless otherwise specified. – T.C. Nov 06 '17 at 20:33

1 Answers1

6

The correct way to do it is to define your distance method in the same namespace as your iter-template, (in this case global namespace).

....
    typename iter::difference_type operator -(const iter &it)
    {
        return ptr - it.ptr;
    }
}; // close template<typename T> class iter

template<typename T>
typename iter<T>::difference_type distance( iter<T> first,  iter<T> last)
{
    std::cout << "my distance called\n";
    return last - first;
}

And later using ADL as seen in this example:

int main()
{
    int arr[5];
    iter<int> it1(&arr[0]);
    iter<int> it2(&arr[5]);

    using std::distance;
    using std::begin;
    using std::end;

    std::cout << distance(it1, it2) << '\n';
    std::cout << "using std::distance\n";
    std::cout << distance(begin(arr), end(arr)) << '\n';

    return 0;
}

will output:

my distance called
5
using std::distance
5

A good explanation of this problem of partial specialization of templates of methods from std is given by Scott Meyers in his book "Effective C++", third edition, item 25.

Bo R
  • 2,334
  • 1
  • 9
  • 17