Find whether file exists or not in HDFS using shell script
You can try -test
option to achieve the same.
hdfs dfs -test -[defszrw] HDFS_PATH
-d
: if the path is a directory, return 0.-e
: if the path exists, return 0.Since 2.7.0
-f
: if the path is a file, return 0.-s
: if the path is not empty, return 0.-r
: if the path exists and read permission is granted, return 0.since 2.8.0
-w
: if the path exists and write permission is granted, return 0.-z
: if the file is zero-length, return 0.
Example:
if hdfs dfs -test -e $HDFS_PATH; then
echo "[$HDFS_PATH] exists on HDFS"
hdfs dfs -ls $HDFS_PATH
fi
Reference: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#test