create iptables rule per process/service

If there is a way to get a process's pid before it starts, then I've never heard about it.

You could write a wrapper which forks first, then adds the rule and execs the process (assuming the program you're running doesn't fork again), since the PID is not changed by the exec(3) call.

/* NOTE this contains zero error checking */
int main(int argc, char **argv) {
    /* Eat argv[0] the name of the wrapper script */
    argv++;
    argc--;

    pid_t my_pid = getpid();

    char *iptables_cmd = NULL;
    asprintf(&iptables_cmd, "/sbin/iptables -A INPUT -m owner --pid_owner %d -j ACCEPT", my_pid);

    system(iptables_cmd);

    execv(argv[0], argv);
}

Building on @Bgs's answer, I would do it like this:

1. Add a new system group, eg. snitch

sudo addgroup --system snitch

2. Add yourself to that group, so that you won't be asked for a password to run processes with the primary group set to it:

sudo adduser $USER snitch

3. Add IPv4 and IPv6 rules to log and reject any packets generated by processes belonging to that group:

sudo iptables  -A OUTPUT -m owner --gid-owner snitch -j LOG --log-prefix 'Snitch: '
sudo ip6tables -A OUTPUT -m owner --gid-owner snitch -j LOG --log-prefix 'Snitch: '
sudo iptables  -A OUTPUT -m owner --gid-owner snitch -j REJECT
sudo ip6tables -A OUTPUT -m owner --gid-owner snitch -j REJECT

4. Open a tail watch on kernel messages:

dmesg -w

5. Launch your target process using sg or any other similar means:

sg snitch 'your target program'

-m owner --pid-owner PID

See http://linuxpoison.blogspot.com/2010/11/how-to-limit-network-access-by-user.html and http://linux.die.net/man/8/iptables

Note that you need the ipt_owner module, as --pid-owner is not supported by xt_owner.

For example (this is just an approximation)

#!/bin/bash
$@ &
iptables -m owner --pid-owner %1 -j REJECT

In reality, though, you're better off using --uid-owner and --gid-owner. First, the --pid-owner criterion only matches the exact pid, meaning your program could easily spawn a child process which would not be blocked by this rule. (At least I haven't read otherwise.) Secondly, iptables(8) warns that --pid-owner is broken on SMP systems (which may or may not apply to you, but in either case limits portability). Third, there is a race condition in the script above, because the process is started before it is blocked. (If there is a way to get a process's pid before it starts, then I've never heard about it.)

It looks like the owner iptables module is that what you want. First, check if it's available in Your system:

iptables -m owner --help

You can read more here: http://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#OWNERMATCH