In Hadoop, how to show current process of -copyFromLocal
Solution 1:
CopyFromLocal
does not have the ability to display the file copy progress. Alternatively, you could open another shell and run the $ watch hadoop fs -ls <filenameyouarecopying>
. This will display the file and its size once every 2.0 seconds.
Solution 2:
It is also possible to track the progress of reading of the local file using pv
command and pipe the file content to hdfs dfs
stdin:
pv mylargefile.txt | hdfs dfs -put - /path/to/file/on/hdfs/mylargefile.txt