Virtual dispatch in a loop

Question

I am wondering if there is a way to optimize case like following:

struct A{
   virtual process(int x) const = 0;
   virtual ~A() = default;
};

void useA(A const &a){
   for(int i = 0; i < 1000; ++i)
      a.process(i);
}

In case like this, if we promise somehow we wont change a vtable, the virtual lookup can be done only once, outside of the loop.

Is there a way to do something like this or result will be very negligible? Or probably there should be some other problem I do not see yet?

If I remember correctly, there is a command line flag on some compilers to tell it that you don't reconstruct another object on the this pointer — JVApen, Nov 24 '20 at 21:10
I think you can use this compilation option: https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-fstrict-vtable-pointers — JVApen, Nov 24 '20 at 21:20
If process takes "long" time, virtual call should be negligible. if process is short, is it really a problem? if yes, maybe change prototype to take range directly. — Jarod42, Nov 24 '20 at 21:28
Whatever dispatch you're using, it's always going to be faster if you build your performance dependent dispatches on chunks of data instead of processing single data points individually. — BeyelerStudios, Nov 24 '20 at 22:24
we or compiler don't know this. May be, may be not. All we know the function is const, but this is not deliverate, I just made it like this when I tested assembly code. — Nick, Nov 25 '20 at 22:39

score 3 · Answer 1 · answered Nov 24 '20 at 21:59

3

The whole reason the vtable gets checked every iteration in loop has to do with a ugly loophole in the language where you can destroy the object the virtual method is called on and recreate another object at the same memory. You can play some ping-pong that way and swap between classes every call of the virtual function.

I hope we all agree that such a code is unacceptable and we should be able to safely assume this doesn't happen in reasonable codebases. If so, you can tell the compiler about this with passing -fstrict-vtable-pointers on the command line of clang. I haven't found if GCC/MSVC supports the same flag.

Regarding the validity of the code I mentioned first, I found some C++ experts talk about this and still be confused, see the twitter conversion

answered Nov 24 '20 at 21:59

JVApen

11,008
5
31
67

A quick benchmark doesn't show any difference with or without the `-fstrict-vtable-pointers` flag: https://godbolt.org/z/YKfx9d – BeyelerStudios Nov 24 '20 at 22:15
1

I should look into that in more detail, I need a bigger screen for that, that's for later – JVApen Nov 24 '20 at 22:21
Looking at your code I don't even see a difference in assembly with this flag enable/disabled. You can see the difference here: https://godbolt.org/z/GqaPsr – JVApen Nov 25 '20 at 19:32
Also if you first do a call on a, it does do something different. https://godbolt.org/z/qj94sv As if it matters that the loop needs to be taken at least once, however. `if n > 0` doesn't cause the same changes. Am currently doubting about removing this answer – JVApen Nov 25 '20 at 19:35

Den-Jason · Answer 2 · 2020-11-24T21:31:02.413

2

Good question!

Firstly, instead of calling the virtual method directly, you could use the polymorphism to return a function, in the form of a functor, std::function or even a lambda, e.g.

struct A{
   virtual std::function<...signature...> getProcessMethod() = 0;
   virtual ~A() = default;
};

See the following resources: Why use functors over functions?

https://www.go4expert.com/articles/cpp-closures-functors-lamdas-stdfunction-t34654/

Once you have your function, you can then apply functional techniques to whatever the range/iterator/vector you're looping round. So you could use a std::transform to perform a map-reduce on the entire set using the function. This can be optimised to use a thread pool etc. How to replicate map, filter and reduce behaviors in C++ using STL?

For further information I wholeheartedly recommend Ivan Cukic's book on Functional Programming with C++ https://www.amazon.co.uk/Functional-Programming-C-Ivan-Cukic/dp/1617293814

edited Nov 24 '20 at 21:31

answered Nov 24 '20 at 21:22

Den-Jason

2,395
1
22
17

3

As the function is `virtual`, return type should have some type-erasure (as `std::function`), resulting to code similar to virtual call. – Jarod42 Nov 24 '20 at 21:24
1

std::function is going to have a ton of overhead compared to a virtual function call, [try it](https://ideone.com/cP2UG0) – BeyelerStudios Nov 24 '20 at 21:34
@BeyelerStudios: [quick-bench](https://quick-bench.com/q/XFD53CvcCP7BtjQ3sBtKNAWIo2M) shows similar result. (but I think regular virtual call should be easier to devirtualize when possible). – Jarod42 Nov 24 '20 at 21:53
@Jarod42 yes clang seems to be similar, even with JVApen's mentioned -fstrict-vtable-pointers flag: https://godbolt.org/z/YKfx9d though gcc and icc run waaay faster on virtual calls. – BeyelerStudios Nov 24 '20 at 22:13
@BeyelerStudios yes I have a suspicion that `std::function` has a lot of overhead - hence why I suggested it as one option of several. It would be interesting to see a proper comparison between that and functor, lambda and function pointer. – Den-Jason Nov 24 '20 at 22:29

Virtual dispatch in a loop

2 Answers2