Run subprocess and print output to logging
You could try to pass the pipe directly without buffering the whole subprocess output in memory:
from subprocess import Popen, PIPE, STDOUT
process = Popen(command_line_args, stdout=PIPE, stderr=STDOUT)
with process.stdout:
log_subprocess_output(process.stdout)
exitcode = process.wait() # 0 means success
where log_subprocess_output()
could look like:
def log_subprocess_output(pipe):
for line in iter(pipe.readline, b''): # b'\n'-separated lines
logging.info('got line from subprocess: %r', line)
I am sure that there is the way to do it without creating temporary file to store process output
You simply have to check for the documentation of Popen
, in particular about stdout
and stderr
:
stdin
,stdout
andstderr
specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values arePIPE
, an existing file descriptor (a positive integer), an existing file object, andNone
.PIPE
indicates that a new pipe to the child should be created. With the default settings ofNone
, no redirection will occur; the child’s file handles will be inherited from the parent. Additionally,stderr
can beSTDOUT
, which indicates that thestderr
data from the child process should be captured into the same file handle as forstdout
.
So you can see that you can either use a file object, or the PIPE
value. This allows you to use the communicate()
method to retrieve the output:
from StringIO import StringIO
process = subprocess.Popen(arguments, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output, error = process.communicate()
log_subprocess_output(StringIO(output))
I'd rewrite your code as:
import shlex
import logging
import subprocess
from StringIO import StringIO
def run_shell_command(command_line):
command_line_args = shlex.split(command_line)
logging.info('Subprocess: "' + command_line + '"')
try:
command_line_process = subprocess.Popen(
command_line_args,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
process_output, _ = command_line_process.communicate()
# process_output is now a string, not a file,
# you may want to do:
# process_output = StringIO(process_output)
log_subprocess_output(process_output)
except (OSError, CalledProcessError) as exception:
logging.info('Exception occured: ' + str(exception))
logging.info('Subprocess failed')
return False
else:
# no exception was raised
logging.info('Subprocess finished')
return True