-6

I recently started adding async support to a library I'm working on, but I hit a slight problem. I started off with something like this (full context later):

return executeRequest<int>(false, d, &callback, false);

That was before adding async support. I attempted to change it to:

return std::async(std::launch::async, &X::executeRequest<int>, this, false, d, &callback, false);

But it failed to compile.

MCVE:

#include <iostream>
#include <future>

int callback(const int& t) {
    std::cout << t << std::endl;   
    return t;
}
class RequestData {
private:
    int x;
public:
    int& getX() {
        return x;   
    }
};
class X {
    public:
        template <typename T>
        T executeRequest(bool method, RequestData& requestData,
                       std::function<T(const int&)> parser, bool write) {
            int ref = 42;
            std::cout << requestData.getX() << std::endl;
            return parser(ref);
        }
        int nonAsync() {
            // Compiles 
            RequestData d;
            return this->executeRequest<int>(false, d, &callback, false);    
        }
        std::future<int> getComments() {
            RequestData d;
            // Doesn't compile 
            return std::async(std::launch::async, &X::executeRequest<int>, this, false, d, &callback, false);
        }
};

int main() {
    X x;
    auto fut = x.getComments();
    std::cout << "end: " << fut.get() << std::endl;
}

And it fails with:

In file included from main.cpp:2:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/5.5.0/../../../../include/c++/5.5.0/future:38:
/usr/bin/../lib/gcc/x86_64-linux-gnu/5.5.0/../../../../include/c++/5.5.0/functional:1505:56: error: no type named 'type' in 'std::result_of<std::_Mem_fn<int (X::*)(bool, RequestData &, std::function<int (const int &)>, bool)> (X *, bool, RequestData, int (*)(const int &), bool)>'
      typedef typename result_of<_Callable(_Args...)>::type result_type;
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/5.5.0/../../../../include/c++/5.5.0/future:1709:49: note: in instantiation of template class 'std::_Bind_simple<std::_Mem_fn<int (X::*)(bool, RequestData &, std::function<int (const int &)>, bool)> (X *, bool, RequestData, int (*)(const int &), bool)>' requested here
          __state = __future_base::_S_make_async_state(std::__bind_simple(
                                                       ^
main.cpp:33:25: note: in instantiation of function template specialization 'std::async<int (X::*)(bool, RequestData &, std::function<int (const int &)>, bool), X *, bool, RequestData &, int (*)(const int &), bool>' requested here
            return std::async(std::launch::async, &X::executeRequest<int>, this, false, d, &callback, false);
                        ^
In file included from main.cpp:2:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/5.5.0/../../../../include/c++/5.5.0/future:38:
/usr/bin/../lib/gcc/x86_64-linux-gnu/5.5.0/../../../../include/c++/5.5.0/functional:1525:50: error: no type named 'type' in 'std::result_of<std::_Mem_fn<int (X::*)(bool, RequestData &, std::function<int (const int &)>, bool)> (X *, bool, RequestData, int (*)(const int &), bool)>'
        typename result_of<_Callable(_Args...)>::type
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~
2 errors generated.

Live example.

The only actual difference between the two (at least that I can see visibly) is that I need to explicitly pass this, because I'm referencing a member function

I played a little around with it, and managed to find that if I replace it with a const RequestData&, it's suddenly allowed. But it instead results in issues elsewhere, because the getter isn't const. At least from what I could find, I need to make it a const function, which is fine for the getter itself, but I also have some setters meaning I can't go with that.

Anyway, I figured I could try std::bind instead. I replaced the async call with:

auto func = std::bind(&X::executeRequest<int>, this, false, d, &callback, false);
return std::async(std::launch::async, func);

And, for some reason, it worked.

The thing that confuses me here, is that it uses the same arguments both times (all three times if you count the non-async variant), and takes the this argument into consideration, given the function I'm calling is a member function.

I dug deeper, and found some alternative solutions (referencing std::thread though), that used std::ref. I know std::async runs std::thread under the hood, so I dug up the documentation:

The arguments to the thread function are moved or copied by value. If a reference argument needs to be passed to the thread function, it has to be wrapped (e.g. with std::ref or std::cref). (emphasis mine)

That makes sense, and explains why it failed. I assume std::async is limited by this as well, and explains why it failed.

However, digging up std::bind:

The arguments to bind are copied or moved, and are never passed by reference unless wrapped in std::ref or std::cref. (emphasis mine)

I don't use std::ref (or if I replace with a const, std::cref) in either, but at least if I understood the documentation right, both of these should fail to compile. The example on cppreference.com also compiles without std::cref (tested in Coliru with Clang and C++ 17).

What's going on here?

If it matters, aside the coliru environment, I originally reproduced the issue in Docker, running Ubuntu 18.04 with Clang 8.0.1 (64 bit). Compiled against C++ 17 in both cases.

Zoe
  • 27,060
  • 21
  • 118
  • 148
  • 1
    Side note for future questions: While your example is reproducible, it should be made minimal if possible, e.g., https://godbolt.org/z/_7gg2Q – Holt Jun 15 '19 at 20:55
  • 2
    @Holt that was as minimal as I got it while keeping it in a runnable state and as close to my actual code as possible. I decided to stick to a class in case that made any difference (though removing it didn't do much IIRC). And my actual code is still bigger than this (and considerably more sensible). Point taken though - thanks – Zoe Jun 15 '19 at 21:02

2 Answers2

5

There is some slight differences in the standard. For std::bind:

Requires: is_­constructible_­v<FD, F> shall be true. For each Ti in BoundArgs, is_­constructible_­v<TDi, Ti> shall be true. INVOKE(fd, w1, w2, …, wN) ([func.require]) shall be a valid expression for some values w1, w2, …, wN, where N has the value sizeof...(bound_­args). The cv-qualifiers cv of the call wrapper g, as specified below, shall be neither volatile nor const volatile.

Returns: An argument forwarding call wrapper g ([func.require]). The effect of g(u1, u2, …, uM) shall be

INVOKE(fd, std::forward<V1>(v1), std::forward<V2>(v2), …, std::forward<VN>(vN))

Where v1, ..., vN have specific types. In your case, what matters is that the stored variable corresponding to d has type std::decay_t<RequestData&> which is RequestData. In this case, you can easily call executeRequest<int> with an lvalue RequestData.

The requirements for std::async are much stronger:

Requires: F and each Ti in Args shall satisfy the Cpp17MoveConstructible requirements, and

INVOKE(decay-copy(std::forward<F>(f)),
   decay-copy(std::forward<Args>(args))...)     // see [func.require], [thread.thread.constr]

The huge difference is decay-copy. For d, you get the following:

decay-copy(std::forward<RequestData&>(d))

Which is a call to the decay-copy function (exposition only), whose return type is std::decay_t<RequestData&>, so RequestData, which is why the compilation fails.


Note that if you used std::ref, the behavior would be undefined since the lifetime of d may ends before the call to executeRequest.

Holt
  • 36,600
  • 7
  • 92
  • 139
  • I'm just slightly confused as to the difference. Is it because `decay-copy` creates a copy which is used in a way that prevents lvalue being an acceptable type for the function, and hence the template? – Zoe Jun 15 '19 at 20:43
  • @Zoe The difference is not really on the feasibility. Under the hood, the version with `std::bind` + `std::async` is probably very similar to the version with only `std::async`, but `std::async` has stronger requirements, likely to avoid surprises when using a function that requires non-`const` references as parameters. – Holt Jun 15 '19 at 20:45
2

The thing that confuses me here, is that it uses the same arguments both times

But it doesn't forward them the same both times. When calling the async version, the callable is invoked as if by calling:

std::invoke(decay_copy(std::forward<Function>(f)), 
            decay_copy(std::forward<Args>(args))...);

The arguments are turned into something akin to temporaries! For this reason, the reference RequestData& requestData cannot bind to its argument. A const reference, a rvalue reference or a plain by value argument would work (as in, bind) here, but a non-const lvalue reference cannot.

std::bind does its invocation differently. It too stores copies, but "the ordinary stored argument arg is passed to the invokable object as lvalue argument[sic]", with the cv qualification of the argument derived from the bind object itself. Since std::bind creates a non-const bind object, the invokable is provided with a non-const lvalue for requestData. The reference binds to that happily.

StoryTeller - Unslander Monica
  • 165,132
  • 21
  • 377
  • 458