c++ std::bad_alloc on std::filesystem::path append

This is caused by a "feature" of Ubuntu, which provides a later libstdc++.so than the one that comes with the system g++. See https://bugs.launchpad.net/ubuntu/+source/gcc-8/+bug/1824721 for more details.

Normally when compiling with GCC 8 the std::filesystem symbols are not present in libstdc++.so and so if you fail to link with -lstdc++fs then you will get a linker error. But because the newer libstdc++.so from GCC 9 does include symbols for std::filesystem, that linker error doesn't happen. Unfortunately, the GCC 9 versions of the filesystem symbols are not compatible with the GCC 8 headers (because the filesystem library was experimental and unstable in GCC 8, and the layout of filesystem::path changed for GCC 9). This means that your program links, but then at runtime it uses the wrong symbols for filesystem::path, and bad things happen.

I didn't anticipate this problem, because I didn't know Ubuntu mixes old libstdc++ headers with a new libstdc++ shared library. That's usually safe to do, except when using "experimental", incomplete features, such as the C++17 features in GCC 8.

The fix I suggested for Ubuntu was to make g++ automatically add -lstdc++fs to the end of your compilation command. If you use any std::filesystem features then the correct definitions for those symbols should be found in GCC 8's libstdc++fs.a (rather than in GCC 9's libstdc++.so) and in most cases everything should Just Work. If Ubuntu didn't update their GCC packages with that workaround yet, you can also make it work by just making sure you manually link with -lstdc++fs (which is documented as required for GCC 8 anyway).

I'll summarize my own findings with what other folks found in the comments. That's not an actual answer (yet), since at this time I cannot explain the reason of the failure.

I was able to reproduce this behavior by installing g++-8 and g++-9 inside a regular ubuntu Docker image, so that I had both /usr/bin/g++-8 and /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.26 available.

According to the gdb stack trace, the error happens somewhere in std::vector constructor. Seems like it happens when the default copy constructor for std::filesystem::path is called inside its operator/:

/usr/include/c++/8/bits/fs_path.h

  /// Append one path to another
  inline path operator/(const path& __lhs, const path& __rhs)
  {
    path __result(__lhs);  // <-- fails here
    __result /= __rhs;
    return __result;
  }

This finding makes it possible to simplify the test case even more:

#include <filesystem>

int main(void)
{
  const std::filesystem::path first = "/tmp";
  const std::filesystem::path second(first);

  return 0;
}

which makes it clear that the problem is somewhere in calling the copy constructor.

The only vector in std::filesystem::path is this vector (presumably, of path components):

/usr/include/c++/8/bits/fs_path.h

    struct _Cmpt;
    using _List = _GLIBCXX_STD_C::vector<_Cmpt>;
    _List _M_cmpts; // empty unless _M_type == _Type::_Multi

According to the stack trace, when copying this vector, we immediately get into stl_vector.h:

/usr/include/c++/8/bits/stl_vector.h

      vector(const vector& __x)
      : _Base(__x.size(),
        _Alloc_traits::_S_select_on_copy(__x._M_get_Tp_allocator()))
      {

but if we print the value of __n in the constructor of _Vector_base here:

      _Vector_base(size_t __n, const allocator_type& __a)
      : _M_impl(__a)
      { _M_create_storage(__n); }

we'll get some insanely large number, which makes me think that an incorrect vector __x was somehow passed down to the copy constructor.

Now, why that happens when you combine g++-8 with the libraries of g++-9, I have no idea (for now) and I'm guessing one should go one level deeper if they need to understand the real reason.

But the answer to your main question, I guess, is "The problem is caused by an incompatibility between your compiler and library versions" :)