Write Python stdout to file immediately
This is happening because normally when process STDOUT is redirected to something other than a terminal, then the output is buffered into some OS-specific-sized buffer (perhaps 4k or 8k in many cases). Conversely, when outputting to a terminal, STDOUT will be line-buffered or not buffered at all, so you'll see output after each \n
or for each character.
You can generally change the STDOUT buffering with the stdbuf
utility:
stdbuf -oL python script.py > log
Now if you tail -F log
, you should see each line output immediately as it is generated.
Alternatively explicit flushing of the output stream after each print should achieve the same. It looks like sys.stdout.flush()
should achieve this in Python. If you are using Python 3.3 or newer, the print
function also has a flush
keyword that does this: print('hello', flush=True)
.
This should do the job:
import time, sys
for i in range(10):
print('bla')
sys.stdout.flush()
time.sleep(5)
As Python will buffer the stdout
by default, here i have used sys.stdout.flush()
to flush the buffer.
Another solution would be to use the -u
(unbuffered) switch of python
. So, the following will do too:
python -u script.py >> log
Variation on the theme of using python's own option for unbuffered output would be to use #!/usr/bin/python -u
as first line.
With #!/usr/bin/env python
that extra argument not gonna work, so alternatively,one could run PYTHONUNBUFFERED=1 ./my_scriipt.py > output.txt
or do it in two steps:
$ export PYTHONUNBUFFERED=1
$ ./myscript.py