Static linking with generated protobufs causes abort

The problem is that your static library contains a file mri.pb.cc which, in its global initializers, is registering type descriptors into the global descriptor database maintained by libprotobuf. Because your static library is loaded into your program twice, this initializer is running twice, but because you have only one copy of libprotobuf in your process, both initializers are registering into the same global database, and it is detecting a conflict.

To fix this problem, you need to change your static library into a shared library, which both the main program and the dynamically-loaded library depend on.

I am not sure why you see different behavior on Windows or OSX. My best guess is that on these platforms, you are actually linking two separate copies of libprotobuf into your program -- one in the main executable and one in the dynamically-loaded library. Thus there are two copies of the descriptor database and no conflict. However, you are likely to see much more subtle problems here. If you ever transfer protobuf object pointers between the main program and the dynamically-loaded module (without serializing and then parsing again) then you could end up having a protobuf object created by one copy of the library but being used with another copy (and therefore a different copy of the descriptor database) which will confuse the library and cause weird things to happen.

Alternatively, if you don't ever pass protobuf objects across the boundary, you might be able to "fix" the problem on Linux by linking libprotobuf statically, in order to get two copies as described above. But this is pretty dangerous; I wouldn't recommend it.

This error appeared for me in the context of an executable linking to two libraries (LibA, LibB) that both happened to compile the same proto message, where LibB depends on LibA and links to it.

I found myself confronted by this situation due to the fact that LibA previously held no dependency on any of the protobuffer framework and LibB built the full set of relevant proto messages for this in-house tooling application to communicate with another application. With a new release of LibA it required new dependencies on two other libraries that compile various proto messages (LibC, LibD). The problem manifested equally from both LibC and LibD, I'll discuss LibC since the solution was identical.

At load time the application loaded LibC, and eventually it came to load the uppermost module LibB and that’s when an abort would be triggered down in LogMessage::Finish() in common.cc. I discovered who was doing this double loading by setting a break point a few levels up from the abort context. Here’s the relevant source to consider for the double loading proto message I'm calling SomeMessage…

void LibC_AddDesc_SomeMessage_2eproto() {
  static bool already_here = false; // <--- Breakpoint Set Here
  if (already_here) return;
  already_here = true;

Breakpoint Hit 1: Loading LibC

LibC.dll!MyNamespace::LibC_AddDesc_SomeMessage_2eproto()  Line 415  C++
LibC.dll!MyNamespace::LibC_AddDesc_ParentMessage_2eproto()  Line 388    C++
LibC.dll!MyNamespace::StaticDescriptorInitializer_ParentMessage_2eproto::StaticDescriptorInitializer_ParentMessage_2eproto()  Line 494  C++
LibC.dll!MyNamespace::`dynamic initializer for 'static_descriptor_initializer_ParentMessage_2eproto_''()  Line 495 + 0x21 bytes C++
msvcr100d.dll!_initterm()  + 0x2c bytes 
LibC.dll!_CRT_INIT(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 289  C
LibC.dll!__DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 506 + 0x13 bytes   C
LibC.dll!_DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 477 C
ntdll.dll!LdrpRunInitializeRoutines()  + 0x1e8 bytes    
ntdll.dll!LdrpInitializeProcess()  - 0x14c9 bytes   
ntdll.dll!string "Enabling heap debug options\n"()  + 0x29a99 bytes 
ntdll.dll!LdrInitializeThunk()  + 0xe bytes 

I could see during the loading of LibC that the breakpoint was hit twice and the static variable already_here was set from false to true, and held at true and would skip the registration of this message.

Breakpoint Hit 2: Loading LibB

When this library attempted its load the variable already_here would be reinitialized to false and we'd proceed to attempt to register this message a second time which triggered the abort.

LibB.dll!MyNamespace::LibC_AddDesc_SomeMessage_2eproto()  Line 415  C++
LibB.dll!MyNamespace::LibC_AddDesc_ParentMessage_2eproto()  Line 388    C++
LibB.dll!MyNamespace::LibC_AddDesc_FullMessage_2eproto()  Line 219  C++
LibB.dll!MyNamespace::StaticDescriptorInitializer_FullMessage_2eproto::StaticDescriptorInitializer_FullMessage_2eproto()  Line 358  C++
LibB.dll!MyNamespace::`dynamic initializer for 'static_descriptor_initializer_FullMessage_2eproto_''()  Line 359 + 0x21 bytes   C++
msvcr100d.dll!_initterm()  + 0x2c bytes 
LibB.dll!_CRT_INIT(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 289  C
LibB.dll!__DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 506 + 0x13 bytes   C
LibB.dll!_DllMainCRTStartup(void * hDllHandle, unsigned long dwReason, void * lpreserved)  Line 477 C
ntdll.dll!LdrpRunInitializeRoutines()  + 0x1e8 bytes    
ntdll.dll!LdrpInitializeProcess()  - 0x14c9 bytes   
ntdll.dll!string "Enabling heap debug options\n"()  + 0x29a99 bytes 
ntdll.dll!LdrInitializeThunk()  + 0xe bytes 

... and we'd wind up in stubs/common.cc at the abort line

void LogMessage::Finish() {
  bool suppress = false;

  if (level_ != LOGLEVEL_FATAL) {
    InitLogSilencerCountOnce();
    MutexLock lock(log_silencer_count_mutex_);
    suppress = internal::log_silencer_count_ > 0;
  }

  if (!suppress) {
    internal::log_handler_(level_, filename_, line_, message_);
  }

  if (level_ == LOGLEVEL_FATAL) {
    abort(); // <----- runtime crash!
  }
}

And to std::err you'd find the following text...

libprotobuf ERROR descriptor_database.cc:57] File already exists in database: SomeMessage.proto
libprotobuf FATAL descriptor.cc:860] CHECK failed: generated_database_->Add(encoded_file_descriptor, size):

The solution was simple, I opened the LibC project and searched for pb and removed those proto messages from LibB. The same was done for LibD.