hdfs Datanode denied communication with namenode because hostname cannot be resolved

You can just change the configure of hdfs-site.xml of namenode Notice the dfs.namenode.datanode.registration.ip-hostname-check


After a lot of issues with this setup, I finally figured what was wrong... Even though my config was right when i set up, it actually happens that resolvconf (the program) tends to reset the /etc/resolv.conf configuration file and overwrite my settings for search domain.local

It also happens that Cloudera and Hadoop use various ways to determine IP address, and unfortunately they are not consistant: Cloudera first looks up IP using SSH, which like PING and other programs use the GLIBC resolver, but later uses HOST, which by passes the GLIBC resolver and uses DNS directly, including the /etc/hosts file, and the /etc/resolv.conf file

So, at first it would work fine, but RESOLVCONF would automatically override my domain and search settings and mess things up.

I ended up REMOVING resolveconf from my setup, and with the proper files in place (hosts, resolv.conf) and making sure HOST resolves to the FQDN, it's all good. So the trick was to remove RESOLVCONF which is installed by default since Ubuntu 10.04 I believe. This is of course true on a local setup like mine. On an actual cluster setup on a network with DNS, just make sure the DNS resolves nodes properly.

Tags:

Hadoop

Hdfs