Error while importing Tensorflow in Python 2.7 in Ubuntu 12.04. 'GLIBC_2.17 not found'

I've just managed to install tensorflow 0.12rc0 on CentOS 6.5 with glibc 2.12, without having root privileges. Simply installing tensorflow binary via pip was giving me an error, related to GLIBC version as well.

Basically, you have 4 options how to deal with this (each with some advantages and disadvantages):

Option 1 - Upgrade your system GLIBC globally.

This is, probably, the best option, if your system supports this, you have root privileges, and you are confident that this upgrade won't break anything for some weird reason. Ultimately, this goes up to upgrading the whole Linux distribution. Here's a nice short list of default GLIBC versions on popular distributions.

Option 2 - Add second GLIBC to your system

Compile or download binary. The most simple&straightforward option. Especially if you only need to run few simple scripts.

  • It is possible to have multiple versions of glibc on the same system, but one should do this with a great care.
  • You won't destroy your system, if all your changes would be limited to a virtual environment.
  • Many programs, installed/compiled before might be relying on old GLIBC, would just crash in your new environment (e.g. your python IDE). Including most basic bash commands, like "lc", "cd", etc.
  • Other side-effects like significant memory leaks are also possible.
  • Thus, it's a very bad idea to add new GLIBC to your normal environment, e.g. via .bashrc.
  • On the other hand, if you need some specific tool in your new virtual environment, you may recompile it, linking against new GLIBC. So, that it would work OK in your new enviroment.
  • However, personally, I quickly gave up recompiling everything I need in a new environment (without root and a package manager).
  • A slightly different approach is officially offered by GLIBC developers, for testing new GLIBC builds.

Option 3 - Patch tensorflow

This may work for TF 0.6.0, but you would probably have to start again from scratch, when each new tensorflow version is released. E.g. here's a fix for 0.9.0.

Option 4 - Compile tensorflow from source

If you re-compile it from source and link against your existing GLIBC, newer GLIBC would be no longer needed. Somehow, this option was not mentioned in any answer here yet. Imho, this is the best option, both "in general", and "specifically for tensorflow".

  • This works OK with r0.11 and would probably work for years, but theoretically, it might break in some newer tensorflow version, if they would decide to actually use some new GLIBC functionality, not present in older versions.
  • To be honest, building tensorflow from source is not straightforward, especially on outdated systems.

A quick summary of "building tensorflow on outdated system":

Although the official guide provides a "installing from sources" section, there are few tricks you need to do to build it on an outdated system. Here I assume, that you do not have root privileges (if you do - you probably would be able to install the same pre-requestities with a package manager, rather them manually building them from source).

I found two well-documented success stories: #1, #2 and a number of useful posts on the official github (mostly about a set of libraries to link inside the binary): #1, #2, #3, #4. I had to combine tricks, described there to successfully compile TF in my case.

  1. First of all, check your gcc --version, and verify that it supports c++11. Mine was 4.4.7, so it won't work. I've downloaded gcc-4.9.4 source code, and compiled it. This step is pretty straightforward, but the compilation itself may take few hours. As a workaround for an issue in bazel, I've compiled gcc with hardcoded paths to as,ld and nm. However, you may try another workarounds: (1, 2).

    #!/bin/sh

unset LIBRARY_PATH CPATH C_INCLUDE_PATH 
unset PKG_CONFIG_PATH CPLUS_INCLUDE_PATH INCLUDE LD_LIBRARY_PATH

cd gcc-4.9.4
./contrib/download_prerequisites

mkdir objdir
cd objdir


# I've added --disable-multilib to fix the following error:
# /usr/bin/ld: crt1.o: No such file: No such file or directory
# collect2: ld returned 1 exit status
# configure: error: I suspect your system does not have 32-bit 
# developement libraries (libc and headers). If you have them,
# rerun configure with --enable-multilib. If you do not have them, 
# and want to build a 64-bit-only compiler, rerun configure 
# with --disable-multilib.           

../configure --prefix=$HOME/opt/gcc-4.9.4 \
             --disable-multilib \
             --disable-nls \
             --enable-languages=c,c++ \
             --with-ld=/usr/bin/ld \
             --with-nm=/usr/bin/nm \
             --with-as=/usr/bin/as

make        
make install

  2. Check your java --version. Bazel requires JDK 8, install it if necessary. (They still provide some jdk7 related downloads, for bazel-0.4.1 but it looks like they consider it deprecated)

  3. I've created a separate use_gcc_4.9.4.sh file, with necessary environment variables. I use source ./use_gcc_4.9.4.sh when I need to so something related to this newer compiler.

    #!/bin/sh
this=$HOME/opt/gcc-4.9.4
export PATH=$this/bin:$PATH
export CPATH=$this/include:$CPATH
export LIBRARY_PATH=$this/lib:$LIBRARY_PATH
export LIBRARY_PATH=$this/lib64:$LIBRARY_PATH
export LD_LIBRARY_PATH=$this/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$this/lib64:$LD_LIBRARY_PATH

  4. The current bazel binary (0.4.1) requires GLIBC 2.14, so we have to compile bazel from source as well (with our new gcc). Works OK, unless you are only allowed to run a very limited number of threads on the target machine. (This post describes some additional workarounds, but in my case they were not needed, maybe due to recent updates in bazel code.)

  5. Obtain tensorflow source code git clone https://github.com/tensorflow/tensorflow, and install prerequisites you need (CUDA,cuDNN,python, etc). See official guide.

  6. If you're not using default system gcc (e.g. if you had to compile newer gcc, like discussed above), add the following linker flags to tensorflow/third_party/gpus/crosstool/CROSSTOOL.tpl, line 59:

    linker_flag: "-L/home/username/localinst/opt/gcc-4.9.4/lib64"
linker_flag: "-Wl,-rpath,/home/username/localinst/opt/gcc-4.9.4/lib64"

    Without this step, you would likely run into error messages like this:

    # ERROR: /home/username/localdistr/src/tensorflow/tensorflow/tensorflow/core/debug/BUILD:33:1: null failed: protoc failed: error executing command bazel-out/host/bin/external/protobuf/protoc '--cpp_out=bazel-out/local_linux-py3-opt/genfiles/' '--plugin=protoc-gen-grpc=bazel-out/host/bin/external/grpc/grpc_cpp_plugin' ... (remaining 8 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
# bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/host/bin/external/protobuf/protoc)
# bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/host/bin/external/protobuf/protoc)
# bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by bazel-out/host/bin/external/protobuf/protoc)

  7. Finally, to avoid GLIBC dependencies, we have to statically link some libraries, by adding the -lrt linker flag (maybe -lm as well). I found multiple posts, suggesting to add this in a different manner:

    • via bazel command line (may sound reasonable, but not working for me on current tensorflow version, somehow),
    • via "bazel-tensorflow/external/protobuf/BUILD"(not sure if it's working, but this does not look convenient - this file is only created during the build attempt itself)

    • via "third_party/gpus/crosstool/CROSSTOOL.tpl" (the same file we've just edited in the previous step, just below the lines we've already added). 

      linker_flag: "-lrt"
linker_flag: "-lm"

    • via "tensorflow/tensorflow.bzl" (works for me, but less convenient just because you have to edit one more file. I'm not sure it's 100% equivalent to the previous point)

    Without -lrt I ran into GLIBC-version-specific error again, trying to import tensorflow:

    # ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/username/anaconda3/envs/myenvname/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow.so)

    Without -lm you may run into this (for me, it turned out to be not necessary).

  8. Run the build process.

    source ./use_gcc_4.9.4.sh
    ./configure
    bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
    bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
    pip install --upgrade /tmp/tensorflow_pkg/tensorflow-0.12.0rc0-cp35-cp35m-linux_x86_64.whl

  1. Try to run the following simple python script to test if the most basic stuff is functioning:

    import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

a = tf.constant(10)
b = tf.constant(32)
print(sess.run(a + b))

I tried BR_User solution and still had an annoying: 

ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found

I am on CentOS 6.7, it also lacks an updated c++ standard lib, so to build on BR_User solution I extracted the correct libstdc++ package, however I found no need for the virtual environment.

Supposing you have already installed tensorflow, it gives:

mkdir ~/my_libc_env
cd ~/my_libc_env
wget http://launchpadlibrarian.net/137699828/libc6_2.17-0ubuntu5_amd64.deb
wget http://launchpadlibrarian.net/137699829/libc6-dev_2.17-0ubuntu5_amd64.deb
wget ftp.riken.jp/Linux/scientific/7.0/x86_64/os/Packages/libstdc++-4.8.2-16.el7.x86_64.rpm
ar p libc6_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
ar p libc6-dev_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
rpm2cpio libstdc++-4.8.2-7mgc30.x86_64.rpm| cpio -idmv

and then run python with:

LD_LIBRARY_PATH="$HOME/my_libc_env/lib/x86_64-linux-gnu/:$HOME/my_libc_env/usr/lib64/" $HOME/my_libc_env/lib/x86_64-linux-gnu/ld-2.17.so `which python`

If it doesn't work, I have another solution, but you won't like it.


I had the same problem, so googling I made these steps:

$ sudo pip install --upgrade virtualenv
$ virtualenv --system-site-packages ~/tensorflow
$ cd ~/tensorflow
$ source bin/activate
$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
$ cd /tmp
$ wget http://launchpadlibrarian.net/137699828/libc6_2.17-0ubuntu5_amd64.deb
$ wget http://launchpadlibrarian.net/137699829/libc6-dev_2.17-0ubuntu5_amd64.deb
$ mkdir libc6_2.17
$ cd libc6_2.17
$ ar p ../libc6_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
$ ar p ../libc6-dev_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
$ cd -
$ LD_LIBRARY_PATH=/tmp/libc6_2.17/lib/x86_64-linux-gnu/ /tmp/libc6_2.17/lib/x86_64-linux-gnu/ld-2.17.so bin/python local/lib/python2.7/site-packages/tensorflow/models/image/mnist/convolutional.py

And to exit:

$ deactivate

That works for me.

Tags:

Python

Ubuntu

Glibc

Tensorflow

Recent Posts