How to trace networking activity of a command?

netstat for simplicity

Using netstat and grepping on the PID or process name:

# netstat -np --inet | grep "thunderbird"
tcp        0      0 192.168.134.142:45348   192.168.138.30:143      ESTABLISHED 16875/thunderbird
tcp        0      0 192.168.134.142:58470   192.168.138.30:443      ESTABLISHED 16875/thunderbird

And you could use watch for dynamic updates:

watch 'netstat -np --inet | grep "thunderbird"'

With:

  • -n: Show numerical addresses instead of trying to determine symbolic host, port or user names
  • -p: Show the PID and name of the program to which each socket belongs.
  • --inet: Only show raw, udp and tcp protocol sockets.

strace for verbosity

You said you tried the strace tool, but did you try the option trace=network? Note that the output can be quite verbose, so you might need some grepping. You could start by grepping on "sin_addr".

 strace -f -e trace=network <your command> 2>&1 | grep sin_addr

Or, for an already running process, use the PID:

 strace -f -e trace=network -p <PID> 2>&1 | grep sin_addr

sysdig allows you to monitor all the activity of the kernel or of several commands running in your system in a go, including and not restricted to network activity.

As the output can be large, you have to build filters, the default page for the most basic filters is quite comprehensible.

It also has the advantage it is not used as an application wrapper as in strace, and it can be quite powerful.

From Sysdig Examples

Networking

See the top processes in terms of network bandwidth usage

sysdig -c topprocs_net 

Show the network data exchanged with the host 192.168.0.1

As binary:

sysdig -s2000 -X -c echo_fds fd.cip=192.168.0.1   

As ASCII:

sysdig -s2000 -A -c echo_fds fd.cip=192.168.0.1 

See the top local server ports:

In terms of established connections:

sysdig -c fdcount_by fd.sport "evt.type=accept"   

In terms of total bytes:

sysdig -c fdbytes_by fd.sport 

See the top client IPs

In terms of established connections

sysdig -c fdcount_by fd.cip "evt.type=accept"   

In terms of total bytes

sysdig -c fdbytes_by fd.cip 

List all the incoming connections that are not served by apache.

sysdig -p"%proc.name %fd.name" "evt.type=accept and proc.name!=httpd"

I'd create a new network namespace, bridge it over to the real network, and then monitor the bridge with tcpdump.