How to disable socket creation for a Linux process, for sandboxing?

ptrace seems to be the most obvious tool but aside from that…

util-linux[-ng] has a command unshare, which uses the kernel's clone/unshare interfaces. If you run the new process throughunshare -n (or clone(CLONE_NEWNET)), any network sockets it creates are in a different namespace. That doesn't solve the kernel resource issue but it does sandbox the process.

The Linux kernel also supports seccomp, a mode entered with prctl(PR_SET_SECCOMP, 1) which prevents the process (well, thread, really) from calling any syscalls other than read, write, exit, and sigreturn. It's a pretty effective sandbox but difficult to use with unmodified code.

You can define a SELinux domain which disallows socket/bind/etc. calls, and perform a dynamic transition into that type. This (obviously) requires a system with an actively enforcing SELinux policy. (Possibly similar things are possible with AppArmor and TOMOYO, but I'm not very familiar with any of them.)

Take a look at systrace - not limited to sockets, but a generic syscall policy generator/enforcer. Quote:

GNU/Linux port is finished and kernel patch is maintained actively by Marius Eriksen. Can be run without kernel changes using the ptrace backend.

Disclamer - I never tried it on Linux.