Cannot run sysctl command in Dockerfile

This is expected since docker restricts access to /proc and /sys (for security). Fundamentally, in order to achieve what you are trying, you need to either give the user CAP_SYS_ADMIN or run in privileged mode, neither of which is allowed during build, see {issue}.

Currently, if you can have those things run after the container is running, then you can use either --cap-add=SYS_ADMIN or --privileged flag. Ideally, these aren't things we would do in a production system, but you seem to be running in a lab setup. If doing it at the run stage, I would recommend first trying the --sysctl flag, but that only supports a subset of command and I'm not sure if it will let you modify kernel settings.

Since Docker containers share the host system's kernel and its settings, a Docker container usually can't run sysctl at all. (You especially can't disable security-critical settings like this one.) You can set a limited number of sysctls on a container-local basis with docker run --sysctl, but the one you mention isn't one of these.

Furthermore, you also can't force changes like this in a Dockerfile. A Docker image only contains a filesystem and some associated metadata, and not any running processes or host-system settings. Even if this RUN sysctl worked, if you rebooted your system and then launched a container from the image, that setting would be lost.

Given what you've shown in this Dockerfile – customized Linux kernel settings, no specific application running, an open-ended ssh daemon as the container process – you might consider whether a virtual machine fits your needs better. You can use a tool like Packer to reproducibly build a VM image in much the same way a Dockerfile builds a Docker image. Since a VM does have an isolated kernel, you can run that sysctl command there and it will work, maybe via normal full-Linux-installation methods like an /etc/sysctl.conf file.