Hadoop HDFS copy with wildcards?
Interesting. This is what I get in my local VM running Hadoop 0.18.0. What version are you using? I can try on 1.2.1 also
hadoop-user@hadoop-desk:~$ hadoop fs -ls /user/hadoop-user/testcopy
hadoop-user@hadoop-desk:~$ hadoop dfs -cp /user/hadoop-user/input/*.txt /user/hadoop-user/testcopy/
hadoop-user@hadoop-desk:~$ hadoop fs -ls /user/hadoop-user/testcopy
Found 2 items
-rw-r--r-- 1 hadoop-user supergroup 79 2014-01-06 04:35 /user/hadoop-user/testcopy/HelloWorld.txt
-rw-r--r-- 1 hadoop-user supergroup 140 2014-01-06 04:35 /user/hadoop-user/testcopy/SampleData.txt
First of all, HDFS copy with wildcards is supported. Secondly, use of hadoop dfs
is deprecated, you'd better use hadoop fs
or hdfs dfs
instead. If you're sure the operation was not successful (although it seems succeed), you could check out the log files of namenode to see what's wrong.
You need use double quote with your path that contains wildcard, like this:
hdfs fs -cp "/path/to/foo*" /path/to/bar/