OSError: [Errno 28] No space left on device Docker, but I have space
As mentioned in the coment by @PeerEZ , this happens when sklearn attempts to parallelize jobs.
sklearn attempts to communicate between processes by writing to /dev/shm, which is limited to 64mb on docker containers.
You can try running with n_jobs=1 as suggested by @PeerEZ (if you can't restart the container), or if parallellization is required, try running the container using the --shm-size
option to set a bigger size for /dev/shm . Eg. -
docker run --shm-size=512m <image-name>
Docker leaves dangling images around that can take up your space. To clean up after docker, run the following:
docker system prune -af
We can use the ‘until’ keyword with the ‘–filter’ option to remove objects that are created before a given timestamp or duration as shown below (objects older than 2 minutes):
docker system prune -a --filter “until=2m”
or in older versions of docker:
docker rm $(docker ps -q -f 'status=exited')
docker rmi $(docker images -q -f "dangling=true")
This will remove exited and dangling images, which hopefully clears out device space.
More reading
Meta: Putting this answer here because it's the top stack-overflow result for that failure and this is a possible fix for it.
If it helps anyone, I received the same error and the problem was that one of my apps log files (laravel.log
) was almost 11GB in size. Deleting that file resolved my problem.