systemctl --user not available for www-data user

The per-user instance of systemd is started by a hook into the login process, a pam_systemd PAM, for both ordinary virtual/real terminal login and remote login via SSH and otherwise.

You are not logging in. You are augmenting the privileges of your existing login session with sudo su www-data. (This is redundant, by the way. sudo -u www-data will go straight to www-data without your running commands as the superuser.) You have not invoked the hook.

Therefore www-data's per-user instance of systemd has not been started, and systemctl --user finds nothing to talk to.

You can start it manually:

% sudo install -d -o www-data /run/user/`id -u www-data`
% sudo systemctl start user@`id -u www-data`

(If you have done these in the wrong order, then stop the service and do them in the right order. Doing them in the wrong order ends up in a state where the runtime directory is empty and lacks the D-Bus and other socket files, but the service is running and will never communicate with clients.)

The one niggle is that your DBUS_SESSION_BUS_ADDRESS variable needs to be changed so that Desktop Bus client programs like systemctl talk to the other account's Desktop Bus broker when you are running them with the privileges of that other account:

% sudo -u www-data DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/`id -u www-data`/bus systemctl --user

This is the simple way. The more complex way is to adjust the PAM configuration so that sudo invokes the pam_systemd hook. However, this has side-effects, particularly with respect to the XDG_RUNTIME_DIR environment variable, that you probably do not desire. Only try this alternative if you are confident that you are alright with the effects of introducing pam_systemd into sudo.

Further reading

• Jonathan de Boyne Pollard (2014). Don't abuse su for dropping user privileges. Frequently Given Answers.
• Lennart Poettering et al. (2017). pam_systemd. systemd manual pages. Freedesktop.org.

So I was able to figure out the missing piece of the puzzle finally. Thanks to some excellent hints from @JdeBP I was able to determine:

• systemd --user was running for www-data
• DBUS_SESSION_BUS_ADRESS seems to be ignored under Ubuntu
• XDG_RUNTIME_DIR was NOT set

Setting XDG_RUNTIME_DIR to export "/run/user/$UID" solved my issue

Steps I followed to get the intended behavior:

% sudo loginctl enable-linger www-data # Enable systemd --user service to start at boot
% XDG_RUNTIME_DIR="/run/user/$UID" systemctl --user # Access services as www-data without actually logging in