Argument list too long for ls
Your error message argument list too long comes from the ***** of ls *.txt
.
This limit is a safety for both binary programs and your Kernel. See ARG_MAX, maximum length of arguments for a new process for more information about it, and how it's used and computed.
There is no such limit on pipe size. So you can simply issue this command:
find -type f -name '*.txt' | wc -l
NB: On modern Linux, weird characters in filenames (like newlines) will be escaped with tools like ls
or find
, but still displayed from *****. If you are on an old Unix, you'll need this command
find -type f -name '*.txt' -exec echo \; | wc -l
NB2: I was wondering how one can create a file with a newline in its name. It's not that hard, once you know the trick:
touch "hello
world"
It depends mainly on your version of the Linux kernel.
You should be able to see the limit for your system by running
getconf ARG_MAX
which tells you the maximum number of bytes a command line can have after being expanded by the shell.
In Linux < 2.6.23, the limit is usually 128 KB.
In Linux >= 2.6.25, the limit is either 128 KB, or 1/4 of your stack size (see ulimit -s
), whichever is larger.
See the execve(2) man page for all the details.
Unfortunately, piping ls *.txt
isn't going to fix the problem, because the limit is in the operating system, not the shell.
The shell expands the *.txt
, then tries to call
exec("ls", "a.txt", "b.txt", ...)
and you have so many files matching *.txt
that you're exceeding the 128 KB limit.
You'll have to do something like
find . -maxdepth 1 -name "*.txt" | wc -l
instead.
(And see Shawn J. Goff's comments below about file names that contain newlines.)
Another workaround:
ls | grep -c '\.txt$'
Even though ls
produces more output than ls *.txt
produces (or attempts to produce), it doesn't run into the "argument too long" problem, because you're not passing any arguments to ls
. Note that grep
takes a regular expression rather than a file matching pattern.
You might want to use:
ls -U | grep -c '\.txt$'
(assuming your version of ls
supports this option). This tells ls
not to sort its output, which could save both time and memory -- and in this case the order doesn't matter, since you're just counting files. The resources spent sorting the output are usually not significant, but in this case we already know you have a very large number of *.txt
files.
And you should consider reorganizing your files so you don't have so many in a single directory. This may or may not be feasible.