Would unused private virtual methods allow future expansion without breaking ABI compatibility?

Question

I'm developing a shared library. Let's say I have the following class definition:

class MyClass {
public:
    //public interface

private:

    virtual void foo1(int);
    virtual void foo2(int, bool);
    virtual void foo3(double);

    virtual void reserved1();
    virtual void reserved2();
    virtual void reserved3();

    class Impl;
    Impl* impl_;
};

The reserved# virtual methods are not overridden in the client code and not called from anywhere. They serve as placeholders for future expansion. Let's say I replace one of the reserved methods with a virtual function with different signature and implementation:

class MyClass {
public:
    //public interface

private:

    virtual void foo1(int);
    virtual void foo2(int, bool);
    virtual void foo3(double);
    virtual void foo4(int, int);

    virtual void reserved2();
    virtual void reserved3();

    class Impl;
    Impl* impl_;
};

It would seem that it achieves full binary compatibility in this way, since the layout of the vtable doesn't change. The problem is that the old code would still ask the dynamic linker to resolve reserved1() and if the definition is not within the library, then the code would crash at link-time, or run-time if someone calls foo4. I assume this issue can't be solved portably, because of ODR. Maybe there's a way to trick the compiler to generate symbol of reserved1 that would act as an alias to foo4?

The point is the same, a linker could treat the link error as non-recoverable and stop the execution of my program. — p12, Feb 19 '12 at 18:58
The compiler doesn't execute your program so saying the link error stops the execution of your code is using the wrong terminology (and is therefore incorrect). — Seth Carnegie, Feb 19 '12 at 19:04
How do you mean that the old code cannot resolve reserved1? You said it doesn't call it. And even if it did, it would be through the vtbl. The client will never link to the implementation of reserved1. It is in fact unknown at compile time. — rasmus, Feb 19 '12 at 23:07
Even if `reserved1` isn't called, it still represents a pointer in the vtbl. If the dynamic linker can't find `reserved1`, the pointer most probably will be set to 0. Now, consider I have two client classes, `C1* c1` and `C2* c2`, compiled with the old and the new definitions of `MyClass`. Let's say I do the following: `MyClass *m1 = c1, *m2 = c2; c1->foo4(); c2->foo4();`. I won't get default behaviour with the old code (`c1->foo4()`), I will get a segfault because of NULL deref! — p12, Feb 20 '12 at 09:09
Have you actually tried this? The part of the vtbl corresponding to MyClass is filled in the constructor of MyClass. This code is recompiled (it is in your shared lib). When MyClass constructor is called from the old client code, the correct vtbl will be setup since it is calling the new constructor in your shared lib. — rasmus, Feb 20 '12 at 12:46
There`s no compatibility issues with `MyClass`. The problem is with `C1`. On construction, `C1::C1()` is called after `MyClass::MyClass()`, so it happily rewrites the vtbl pointer with a value pointing to the old vtbl which is somewhere in the old client code. That code hasn't been recompiled, so the old vtbl will have 0 in the cell for `reserved1` / `foo4`, because it asks the dynamic linker for `reserved1`. So if I call `foo4` I get segfault. — p12, Feb 20 '12 at 18:28
A virtual table won't have 0 in it. Either you declare the method as abstract (pure virtual, with =0), in which case this class does not have a virtual table - since it cannot be instantiated, or you don't - in which case the linker will force you to provide some kind of implementation. — Asaf, Feb 20 '12 at 19:17
@Asaf: The code is already compiled and expects me to provide a shared library with the implementation of the function. If my library doesn't have the implementation, the dynamic linker either fails or puts 0 in the vtable as a fallback. — p12, Feb 20 '12 at 19:20

Seth Carnegie · Answer 1 · 2012-02-19T19:01:57.270

2

You don't need to do this because, as long as the order of the methods is not altered, you can add methods to the end of the vtable without changing the beginning; pointers are accessed in the vtable by their offset from the beginning, so adding something to the end won't affect anything.

This is the entire point of interface classes like this: you can pass pointers to derived classes, which have a vtable with extra methods on the end, to functions which expect a pointer to a base class.

For instance:

// Class:

class MyClass {
public:
    //public interface

private:

    virtual void foo1(int);
    virtual void foo2(int, bool);
    virtual void foo3(double);

    class Impl;
    Impl* impl_;
};

// current vtable:

+-------------------+
| foo1(int)         | < offset 0
| foo2(int, bool)   | < offset 1
| foo3(double)      | < offset 2
+-------------------+

// code is compiled and references offsets 1 and 2 in the vtable

// then you change the class with an added method:

class MyClass {
public:
    //public interface

private:

    virtual void foo1(int);
    virtual void foo2(int, bool);
    virtual void foo3(double);
    virtual void foo4(int, int);

    class Impl;
    Impl* impl_;
};

// New vtable:

+-------------------+
| foo1(int)         | < offset 0
| foo2(int, bool)   | < offset 1
| foo3(double)      | < offset 2
| foo4(int, int)    | < offset 3
+-------------------+

// the offsets of the first three are the same so the old code
// that was compiled to use offsets 1 and 2 still works

edited Feb 19 '12 at 19:01

answered Feb 19 '12 at 18:54

Seth Carnegie

73,875
22
181
249

1

But this wouldn't preserve the ABI. The sizeof a MyClass object would change. – Oliver Charlesworth Feb 19 '12 at 18:55
1

@Seth: What if the class is inherited in the client class, which adds some virtual methods of its own? Your solution won't work, since client virtual methods would be placed in the same locations as my new methods. – p12 Feb 19 '12 at 18:59
1

@OliCharlesworth the size of a class doesn't change when you add a method to it does it? – Seth Carnegie Feb 19 '12 at 19:00
@grumm143 I don't understand what "What if the class is inherited in the client class, which adds some virtual methods of its own?" means. However, if a child class adds some virtual methods, then the virtual methods will be added to the end of the vtable, not disrupting the offset of the base class function offsets. – Seth Carnegie Feb 19 '12 at 19:01
@Seth: Oh, yes, I suppose it's only the size of the vtable that would change, not the size of the individual objects. Of course, a bigger vtable could force other stuff to be shifted in memory, leading to problems.. – Oliver Charlesworth Feb 19 '12 at 19:01
@OliCharlesworth what would it shift? I wasn't aware that things were stored next to the vtable so that the size of the vtable would change their offset. I actually doubt this because I have written DLLs that used outdated interface headers (the up-to-date interface had more virtual methods) and it posed no problem (though I doubt my doubt because it's personal experience (and the compiler was VC++...)). That's the entire point of polymorphism I thought. But I've thought wrong before :) – Seth Carnegie Feb 19 '12 at 19:05
1

@Seth: Ok. Let's say I have `class Client : public MyClass { <...> virtual void clientfoo(); };` It only has vtable for 4 functions. If I add another virtual method to `MyClass`, I need to recompile `Client`. – p12 Feb 19 '12 at 19:06
@Seth: I guess it depends on the platform in question. But in theory, anything that's resolved at link-time, e.g. functions or global variables. – Oliver Charlesworth Feb 19 '12 at 19:06
1

@OliCharlesworth: vtable is implemented within a class as a hidden pointer. Size of that pointer doesn't change, obviously. – SigTerm Feb 19 '12 at 19:31
@grumm143 Yes, having a virtual method in a derived class that is not recompiled will break the ABI, when the base class adds another virtual method. – rasmus Feb 19 '12 at 22:52

score 2 · Accepted Answer · answered Feb 20 '12 at 19:05

2

Since the function reserved1 is only there to preserve vtable layout compatibility, presumably nothing in the client code will call it.

If it isn't called client code doesn't need any linker reference to it: this is obviously all platform-specific, but in general your scheme should work fine.

Are the virtual methods really private though? If they can't be called or overridden from the client, you could just expose an opaque forward declaration and keep the implementation entirely inside your dynamic lib (eg, MyClass::PImpl).

answered Feb 20 '12 at 19:05

Useless

64,155
6
88
132

`private` means just that they can't be called directly. Client code can override them. Only the `reserved#` function would not be overridden. One good feature of `private` is that this guarantees that the call would never be inlined, as the client could only call these through public nonvirtual functions. So I know that I can add implementations of virtual functions throughout my class hierarchy. – p12 Feb 20 '12 at 19:08
Should work then. Have you tried a toy example and actually found a linker ref to `reserved1` in the client? – Useless Feb 20 '12 at 19:14
Yes. GCC generates a vtable in the client executable. – p12 Feb 20 '12 at 19:22

score 0 · Answer 3 · answered Feb 21 '12 at 15:21

You can use reserved virtual functions at end of class declaration, but your class should has an exported constructor or a factory method that can return a pointer to created class from the shared library side:

    class MyClass {
    public:
        MyClass(...);
        ...
    };

OR

    class MyClass {
    public:
        MyClass* create(...);
        ...
    };

Your sample class MyClass has no any declared constructor, so the compiler (GCC) will generate inline constructor for this class automatically. This constructor will create old v-table for old client applications, which has no entry for foo4.

If you declare a factory method then it will return pointer to a class object with new v-table for old clients, so that they will be able to find foo4 and run your new code.

Won't work. Any client class, that inherits from `MyClass`, would still be constructed in the client code. That means old vtable would be used. — p12, Feb 21 '12 at 18:41
@grumm143 - Is it an option for you to deny inheritance through the declaring of private constructor only and creating a public factory method for class users? — linuxbuild, Feb 22 '12 at 08:51

Would unused private virtual methods allow future expansion without breaking ABI compatibility?

3 Answers3

Linked