In Hadoop, how to show current process of -copyFromLocal

Solution 1:

CopyFromLocal does not have the ability to display the file copy progress. Alternatively, you could open another shell and run the $ watch hadoop fs -ls <filenameyouarecopying>. This will display the file and its size once every 2.0 seconds.

Solution 2:

It is also possible to track the progress of reading of the local file using pv command and pipe the file content to hdfs dfs stdin:

pv mylargefile.txt | hdfs dfs -put - /path/to/file/on/hdfs/mylargefile.txt

Tags:

Hadoop

Hdfs