How do I get sudo -u $user to use the users env?

Try sudo -i -u $user

gerald@book:~$ env |grep HOME
HOME=/home/gerald
gerald@book:~$ sudo -u ubuntu env |grep HOME
HOME=/home/gerald
gerald@book:~$ sudo -i -u ubuntu env |grep HOME
HOME=/home/ubuntu

man sudoers on Debian mentions another possibility. Not sure which way around you want, but your question sounds like you would want to have the env_reset option from /etc/sudoers - the opposite is basically the env_keep list. In order to set the proper HOME you can use the -H option to sudo directly or, again in sudoers, with the always_set_home option.

Alternatively you could use env_file to specify an exact environment you want to pass. However, I think it is best if you check out the env_* options from man sudoers, because /etc/sudoers controls it all and that's the point to turn to.

Here's part of the context in which I use env_reset inside my sudoers file:

Defaults        !lecture
Defaults        env_reset
Defaults        syslog=auth
Defaults        log_year