Vector Deleting Destructor and Weak Linkage

Now that the discussions on weak linker symbols and vector deleting destructors are in place, it is time to discuss a fact that might seem esoteric but has far reaching implications. After that, it is time to ask for your help.

In VC++, Vector deleting destructors are defined with weak linkage at the translation unit that defined the class, and strong linkage at any translation unit that calls new[] on the class.

Say what?

The first part of this statement (v-d-dtors have weak linkage) was already demonstrated at the post on weak linkage – given any cpp file which defines a non trivial class, you can dumpbin its obj file and see for yourself.

Now some code to demonstrate the full statement:

//C.h
struct C
{
  virtual ~C();
};

//C.cpp
#include "C.h"
C::~C() {} 

//D.h
struct D
{
void Func();
};

//D.cpp
#include "D.h"
#include "C.h"
void D::Func()
{
  C* = new C[42];
}

A dumpbin of C.obj shows:

017 00000000 UNDEF  notype ()    External     | ??3@YAXPAX@Z (void __cdecl operator delete(void *))
018 00000000 SECT4  notype ()    External     | ??1C@@UAE@XZ (public: virtual __thiscall C::~C(void))
019 00000000 SECT6  notype ()    External     | ??_GC@@UAEPAXI@Z (public: virtual void * __thiscall C::`scalar deleting destructor'(unsigned int))
01A 00000000 UNDEF  notype ()    WeakExternal | ??_EC@@UAEPAXI@Z (public: virtual void * __thiscall C::`vector deleting destructor'(unsigned int))

While a dumpbin of D.obj shows:

01D 00000000 UNDEF  notype ()    External     | ??_L@YGXPAXIHP6EX0@Z1@Z (void __stdcall `eh vector constructor iterator'(void *,unsigned int,int,void (__thiscall*)(void *),void (__thiscall*)(void *)))
01E 00000000 UNDEF  notype ()    External     | ??_M@YGXPAXIHP6EX0@Z@Z (void __stdcall `eh vector destructor iterator'(void *,unsigned int,int,void (__thiscall*)(void *)))
01F 00000000 UNDEF  notype ()    External     | ??2@YAPAXI@Z (void * __cdecl operator new(unsigned int))
020 00000000 UNDEF  notype ()    External     | ??3@YAXPAX@Z (void __cdecl operator delete(void *))
021 00000000 SECT8  notype ()    External     | ?Func@D@@QAEXXZ (public: void __thiscall D::Func(void))
022 00000000 UNDEF  notype ()    External     | ??1C@@UAE@XZ (public: virtual __thiscall C::~C(void))
023 00000000 SECT4  notype ()    External     | ??0C@@QAE@XZ (public: __thiscall C::C(void))
024 00000000 SECT6  notype ()    External     | ??_EC@@UAEPAXI@Z (public: virtual void * __thiscall C::`vector deleting destructor'(unsigned int))

What this means is that to successfully complete the linkage of C.obj, the linker must now load D.obj – because both contain implementations of the same function, but C defines a weak external implementation and D defines a strong external implementation (of a C method!).

Ok, that’s kinda weird, but why should I care?

Here’s why:

What happens when C.cpp and D.cpp are part of a static library?

Unlike executables (.exe or .dll), when processing a static lib the linker only loads obj files that are referenced, i.e., whose contents are needed for successful linkage. Once loaded, an obj file must have it’s contents successfully link (unless you’re building with /GL, but let’s ignore that here). Let’s expand the previous example a bit :

//main.cpp
#include "StaticLib\C.h"

int main(int, char)
{
  C c;
  return 0;
}

//StaticLib\C.h
struct C
{
  virtual ~C();
};

//StaticLib\C.cpp
#include "C.h"
C::~C() {} 

//StaticLib\D.h
struct D
{
  void Func();
};

//StaticLib\D.cpp
#include "D.h"
#include "C.h" 

extern void SomeJunkImplementedElsewhere();
void D::Func()
{
  C* = new C[42];
  SomeJunkImplementedElsewhere();
}

Can you already see what happens now?

Now for the program to successfully build you must satisfy D.cpp’s linkage – which means dragging in another library – although you never consumed D’s functionality in the first place.

I wish this was just a theoretical peculiarity. The solutions I’m working on consist of a complicated network of literally hundreds of static libraries, and time and time again we find ourselves forced to drag in weird dependencies that the code we actually run never uses.  It seems unbelievable, but almost all of these unexplainable dependencies boil down to this esoteric fact – vector deleting destructors have weak linkage at the point of class definition.

That was nice. Now go and report it.

I did. Over half a year ago.   The report was originally closed as ‘By Design’, and after an explicit request the following explanation from Karl Niu arrived:

To explain the “By Design” resolution, imagine that you have “new A[n]” and “delete[] pA” in different translation units. In such a case, the compiler needs to define the strong external in the translation unit containing the “new A[n]”.

Which I just don’t understand: the weak/strong debate is not over new[] or delete[], but rather over vector deleting destructors, which are not user-overridable in the first place. Wherever delete[] is overloaded, it should be able to fetch the vector-deleting-dtor from the translation unit that defined it – hopefully, the one that defined the class it’s deleting.   I tried to ask again, twice, and got no response for 6 months now.

Now, I regularly report many bugs at MS Connect, almost all of which never get resolved (which I can live with. I’m doing this mostly in hope of helping fellow devs googling their trouble) – but this one leaves me frustrated. It feels as if despite my best efforts I failed to clearly communicate the issue.    It seems like an esoteric technicality, yet it actively hinders decoupling – thereby damaging large software systems at the architecture level!

Why golly Ofek, that’s really bad. But what can I do?

You can either –

(1) Dig in and tell me in the comments where I’m wrong.  It was initially resolved as ‘by design’, and even got an explanation (sorta), so I might be missing some valid reason for this sorry state of affairs.

(2) Go to the bug page and upvote it.  This one realy deserves attention from the VC++ team.

But I urge you to do either.  Thanks!

red-pill-or-blue-pill

Advertisements
This entry was posted in VC++. Bookmark the permalink.

9 Responses to Vector Deleting Destructor and Weak Linkage

  1. Marco A. says:

    Interesting point, I didn’t dive deeper into the issue but sometimes I noticed the “dragging of unused libraries” phenomenon too. And as a sidenote this might also help rendering the linker faster. +1

  2. SergeyN says:

    So, what’s the workaround ? use std::vector ? custom inlined my_new ? or ?

    • Ofek Shilon says:

      This is compiler-level treatment of the new keyword, and short of rolling your own memory manager I do not know of a workaround. (‘my_new’ doesn’t have to be inlined). (std::vector uses new internally)
      I understand this as a design bug by MS, whose fix would now be considered a ‘breaking change’ and thereby require considerable persuasion. Hope the votes would amount to it.

    • Ofek Shilon says:

      @SergeyN: as @JoeH points out there is an ugly-ish workaround – add to C.cpp a dummy call to new C[N]. This drags in strong vector dtor, which overrides the need for the linker to load D.obj.

  3. Joe H. says:

    The response from MS makes a certain degree of sense, when considering the possibility of ODR violations.
    Their mechanism would seem to favor selection of the ‘delete[]‘ from the same translation-unit as the ‘new[]‘, thus hopefully mitigating certain disasters that might occur from any ODR violations. I’m not saying it’s the best solution, but I can kinda see why one might want to implement it the way they did.
    In your example above, does it mitigate the disaster if you add a bogus helper-method inside ‘StaticLib\C.cpp’, which invokes a “new C[x]“? If I understand the linker behaviour, this should cause the C.cpp to have an ‘External’ reference rather than ‘WeakExternal’, which ought to prevent ‘D.obj’ from being spuriously pulled in?

    • Ofek Shilon says:

      @JoeH – thanks for the reply!
      I might still be missing something. (1) Why should linkage of new[] and delete[] be any different? As a guideline, all user-overridable, compiler-generated functions should be weak-externals, regardless of the translation unit. Any user given overload should be strong external. That’s the only consistent way I can think of. (2) Note that the function under discussion is *not* delete[], but rather an internal delete[] helper, which is *not* user-overloadable.

    • Ofek Shilon says:

      And sorry: yes, adding a bogus ‘new C[N]’ mitigate the disaster. I included it in the original project I uploaded to MS Connect.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s