Too many open files while ensure index mongo
I added a temporary ulimit -n 4096
before the restore command.
also you can use
mongorestore --numParallelCollections=1 ...
and that seems to help.
But still the connection pool seems to get exhausted.
NB: This solution does/may not work with recent Mac OSs (comments indicate >10.13?). Apparently, changes have been made for security purposes.
Conceptually, the solution applies - following are a few sources of discussion:
- https://wilsonmar.github.io/maximum-limits/
- https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c
- https://superuser.com/questions/433746/is-there-a-fix-for-the-too-many-open-files-in-system-error-on-os-x-10-7-1
--
I've had the same problem (executing a different operation, but still, a "Too many open files" error), and as lese says, it seems to be down to the 'maxfiles' limit on the machine running mongod.
On a mac, it is better to check limits with:
sudo launchctl limit
This gives you:
<limit name> <soft limit> <hard limit>
cpu unlimited unlimited
filesize unlimited unlimited
data unlimited unlimited
stack 8388608 67104768
core 0 unlimited
rss unlimited unlimited
memlock unlimited unlimited
maxproc 709 1064
maxfiles 1024 2048
What I did to get around the problem was to temporarily set the limit higher (mine was originally something like soft: 256, hard: 1000 or something weird like that):
sudo launchctl limit maxfiles 1024 2048
Then re-run the query/indexing operation and see if it breaks. If not, and to keep the higher limits (they will reset when you log out of the shell session you've set them on), create an '/etc/launchd.conf' file with the following line:
limit maxfiles 1024 2048
(or add that line to your existing launchd.conf file, if you already have one).
This will set the maxfile via launchctl on every shell at login.