Can I use systemd to start and stop a service based on the presence of a file?

I would try this. Create an additional service using PathChanged:

foo-stop.path

[Path] 
PathChanged=/tmp/foo.path

[Install] 
WantedBy=multi.user.target

Then create: foo-stop.service

Have it's "ExecStart" script check to see if /tmp/foo.path was deleted (since PathChanged could fire on other changes as well). If the path has been removed, have the script call /bin/systemctl stop foo.