YARN Application exited with exitCode: -1000 Not able to initialize user directories

Actually this is due to the permission issues on some of the yarn local directories. I started using LinuxContainerExecutor (in non secure mode with nonsecure-mode.local-user as kailash) and made corresponding changes. However due to some (unknown) reason NodeManager failed to clean local directories for users, and there still existed directories with previous user (in my case yarn).

So to solve this, I first had to find the value of the property yarn.nodemanager.local-dirs (with Cloudera use search option to find this property for YARN service, otherwise look into yarn-site.xml in hadoop conf directory), and then delate the files/directories under usercache for all the node manager nodes. In my case, I used:

rm -rf /yarn/nm/usercache/*

Just in case if someone is lost with usercache location. If you don't have the yarn.nodemanager.local-dirs configured anywhere, look for it in the default location ${hadoop.tmp.dir}/nm-local-dir. Again, if the hadoop.tmp.dir is not configured in the core-site.xml, it will be under /tmp/hadoop-${user.name}. The user.name is the UNIX user that you are using to run the current Hadoop process. All the configuration file are under $HADOOP_INSTALL/etc/hadoop/ by default.