Calling the "source" command from subprocess.Popen
source
is not an executable command, it's a shell builtin.
The most usual case for using source
is to run a shell script that changes the environment and to retain that environment in the current shell. That's exactly how virtualenv works to modify the default python environment.
Creating a sub-process and using source
in the subprocess probably won't do anything useful, it won't modify the environment of the parent process, none of the side-effects of using the sourced script will take place.
Python has an analogous command, execfile
, which runs the specified file using the current python global namespace (or another, if you supply one), that you could use in a similar way as the bash command source
.
You could just run the command in a subshell and use the results to update the current environment.
def shell_source(script):
"""Sometime you want to emulate the action of "source" in bash,
settings some environment variables. Here is a way to do it."""
import subprocess, os
pipe = subprocess.Popen(". %s; env" % script, stdout=subprocess.PIPE, shell=True)
output = pipe.communicate()[0]
env = dict((line.split("=", 1) for line in output.splitlines()))
os.environ.update(env)
Broken Popen("source the_script.sh")
is equivalent to Popen(["source the_script.sh"])
that tries unsuccessfully to launch 'source the_script.sh'
program. It can't find it, hence "No such file or directory"
error.
Broken Popen("source the_script.sh", shell=True)
fails because source
is a bash builtin command (type help source
in bash) but the default shell is /bin/sh
that doesn't understand it (/bin/sh
uses .
). Assuming there could be other bash-ism in the_script.sh
, it should be run using bash:
foo = Popen("source the_script.sh", shell=True, executable="/bin/bash")
As @IfLoop said, it is not very useful to execute source
in a subprocess because it can't affect parent's environment.
os.environ.update(env)
-based methods fail if the_script.sh
executes unset
for some variables. os.environ.clear()
could be called to reset the environment:
#!/usr/bin/env python2
import os
from pprint import pprint
from subprocess import check_output
os.environ['a'] = 'a'*100
# POSIX: name shall not contain '=', value doesn't contain '\0'
output = check_output("source the_script.sh; env -0", shell=True,
executable="/bin/bash")
# replace env
os.environ.clear()
os.environ.update(line.partition('=')[::2] for line in output.split('\0'))
pprint(dict(os.environ)) #NOTE: only `export`ed envvars here
It uses env -0
and .split('\0')
suggested by @unutbu
To support arbitrary bytes in os.environb
, json
module could be used (assuming we use Python version where "json.dumps not parsable by json.loads" issue is fixed):
To avoid passing the environment via pipes, the Python code could be changed to invoke itself in the subprocess environment e.g.:
#!/usr/bin/env python2
import os
import sys
from pipes import quote
from pprint import pprint
if "--child" in sys.argv: # executed in the child environment
pprint(dict(os.environ))
else:
python, script = quote(sys.executable), quote(sys.argv[0])
os.execl("/bin/bash", "/bin/bash", "-c",
"source the_script.sh; %s %s --child" % (python, script))