Redirect stdout to a file only for a specific thread
This answer allow redirecting to specific file, but also to have multiple outputs (sys.stdout + file), and also to redirect main thread logs to each thread filelog (which was very useful in my case).
First of all, let's define a class for new redirection :
class SysRedirect(object):
def __init__(self):
self.terminal = sys.stdout # To continue writing to terminal
self.log={} # A dictionary of file pointers for file logging
def register(self,filename): # To start redirecting to filename
ident = threading.currentThread().ident # Get thread ident (thanks @michscoots)
if ident in self.log: # If already in dictionary :
self.log[ident].close() # Closing current file pointer
self.log[ident] = open(filename, "a") # Creating a new file pointed associated with thread id
def write(self, message):
self.terminal.write(message) # Write in terminal (comment this line to remove terminal logging)
ident = threading.currentThread().ident # Get Thread id
if ident in self.log: # Check if file pointer exists
self.log[ident].write(message) # write in file
else: # if no file pointer
for ident in self.log: # write in all thread (this can be replaced by a Write in terminal)
self.log[ident].write(message)
def flush(self):
#this flush method is needed for python 3 compatibility.
#this handles the flush command by doing nothing.
#you might want to specify some extra behavior here.
pass
Then, I only need to initialise in my main thread
sys.stdout=SysRedirect()
Then in each thread, I only have to register and specifying a filename
sys.stdout.register('threadX.log')
And in main Thread, I can redirect for example to :
sys.stdout.register('mainthread.log')
But in my case I prefer not to register the main Thread so all std in main thread is written in all others logs
I stumbled across this post while searching for a way to do exactly this. I wanted to make an interactive Python console that uses ajax to proxy requests to a server, and return all output for only the thread executing. I ended up figuring it out and wanted to share my solution.
There's a class that comes with the werkzeug
python library called local.LocalProxy
that can make a module-level function behave like a property. For example, this will make sys.stdout
behave exactly as normal, but it will be proxied through the LocalProxy
class.
import sys
import werkzeug
sys.stdout = werkzeug.local.LocalProxy(lambda: sys.stdout)
Expanding on this, I then wrote a function in place of the lambda
above to return a StringIO
object if it's a different thread:
import threading
import sys
import cStringIO
import werkzeug
thread_proxies = {}
def redirect():
ident = threading.currentThread().ident
thread_proxies[ident] = cStringIO.StringIO()
return thread_proxies[ident]
def proxy():
ident = threading.currentThread().ident
return thread_proxies.get(ident, sys.stdout)
sys.stdout = werkzeug.local.LocalProxy(proxy)
And then in any thread I want redirected, I can just call:
string_io = redirect()
And all of the output that would go to sys.stdout
is instead now written to the StringIO
object.
But wait! I need to capture sys.stdout
, sys.__stdout__
, sys.stderr
, and sys.__stderr__
, so I wrote this library, which I called stdout_helpers
in my codebase:
import threading
import sys
import cStringIO
from werkzeug import local
# Save all of the objects for use later.
orig___stdout__ = sys.__stdout__
orig___stderr__ = sys.__stderr__
orig_stdout = sys.stdout
orig_stderr = sys.stderr
thread_proxies = {}
def redirect():
"""
Enables the redirect for the current thread's output to a single cStringIO
object and returns the object.
:return: The StringIO object.
:rtype: ``cStringIO.StringIO``
"""
# Get the current thread's identity.
ident = threading.currentThread().ident
# Enable the redirect and return the cStringIO object.
thread_proxies[ident] = cStringIO.StringIO()
return thread_proxies[ident]
def stop_redirect():
"""
Enables the redirect for the current thread's output to a single cStringIO
object and returns the object.
:return: The final string value.
:rtype: ``str``
"""
# Get the current thread's identity.
ident = threading.currentThread().ident
# Only act on proxied threads.
if ident not in thread_proxies:
return
# Read the value, close/remove the buffer, and return the value.
retval = thread_proxies[ident].getvalue()
thread_proxies[ident].close()
del thread_proxies[ident]
return retval
def _get_stream(original):
"""
Returns the inner function for use in the LocalProxy object.
:param original: The stream to be returned if thread is not proxied.
:type original: ``file``
:return: The inner function for use in the LocalProxy object.
:rtype: ``function``
"""
def proxy():
"""
Returns the original stream if the current thread is not proxied,
otherwise we return the proxied item.
:return: The stream object for the current thread.
:rtype: ``file``
"""
# Get the current thread's identity.
ident = threading.currentThread().ident
# Return the proxy, otherwise return the original.
return thread_proxies.get(ident, original)
# Return the inner function.
return proxy
def enable_proxy():
"""
Overwrites __stdout__, __stderr__, stdout, and stderr with the proxied
objects.
"""
sys.__stdout__ = local.LocalProxy(_get_stream(sys.__stdout__))
sys.__stderr__ = local.LocalProxy(_get_stream(sys.__stderr__))
sys.stdout = local.LocalProxy(_get_stream(sys.stdout))
sys.stderr = local.LocalProxy(_get_stream(sys.stderr))
def disable_proxy():
"""
Overwrites __stdout__, __stderr__, stdout, and stderr with the original
objects.
"""
sys.__stdout__ = orig___stdout__
sys.__stderr__ = orig___stderr__
sys.stdout = orig_stdout
sys.stderr = orig_stderr
And now at the start of my app I call:
stdout_helpers.enable_proxy()
And in any thread I now call:
string_io = stdout_helpers.redirect()