How do I connect a veth device inside an 'anonymous' network namespace to one outside?

ip link has a namespace option, which in addition to a network namespace name, can use a PID to refer a process' namespace. If PID namespaces are shared between the processes, you can move devices either way; it is probably easiest from inside, when you consider PID 1 being "outside". With separate PID namespaces you need to move from outer (PID) namespace to the inner one.

For example, from inside of a network namespace you can create a veth device pair to PID 1 namespace:

ip link add veth0 type veth peer name veth0 netns 1

How namespaces work in Linux

Every process has reference files for their namespaces in /proc/<pid>/ns/. Additionally, ip netns creates persistent reference files in /run/netns/. These files are used with setns system call to change the namespace of the running thread to a namespace pointed by such file.

From shell you can enter to another namespace using nsenter program, providing namespace files (paths) in arguments.

A good overview of Linux namespaces is given in the Namespaces in operation article series on LWN.net.

Setting up namespaces

When you set up multiple namespaces (mount, pid, user, etc.), set up network namespace as early as possible, before altering mount and pid namespaces. If you do not have shared mount or pid namespaces, you do not have any way to point to the network namespace outside, because you can not see the files referring to network namespaces outside.

If you need more flexibility than the command line utilities provide, you need to use the systemcalls to manage name spaces directly from your program. For documentation, see the relevant man pages: man 2 setns, man 2 unshare and man 7 namespaces.