python: is it possible to attach a console into a running process

If you have access to the program's source-code, you can add this functionality relatively easily.

See Recipe 576515: Debugging a running python process by interrupting and providing an interactive prompt (Python)

To quote:

This provides code to allow any python program which uses it to be interrupted at the current point, and communicated with via a normal python interactive console. This allows the locals, globals and associated program state to be investigated, as well as calling arbitrary functions and classes.

To use, a process should import the module, and call listen() at any point during startup. To interrupt this process, the script can be run directly, giving the process Id of the process to debug as the parameter.

Another implementation of roughly the same concept is provided by rconsole. From the documentation:

rconsole is a remote Python console with auto completion, which can be used to inspect and modify the namespace of a running script.

To invoke in a script do:

from rfoo.utils import rconsole
rconsole.spawn_server()

To attach from a shell do:

$ rconsole

Security note: The rconsole listener started with spawn_server() will accept any local connection and may therefore be insecure to use in shared hosting or similar environments!


Use pyrasite-shell. I can't believe it works so well, but it does. "Give it a pid, get a shell".

$ sudo pip install pyrasite
$ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope # If YAMA activated, see below.
$ pyrasite-shell 16262
Pyrasite Shell 2.0
Connected to 'python my_script.py'
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> globals()
>>> print(db_session)
>>> run_some_local_function()
>>> some_existing_local_variable = 'new value'

This launches the python shell with access to the globals() and locals() variables of that running python process, and other wonderful things.

Only tested this personally on Ubuntu but seems to cater for OSX too.

Adapted from this answer.

Note: The line switching off the ptrace_scope property is only necessary for kernels/systems that have been built with CONFIG_SECURITY_YAMA on. Take care messing with ptrace_scope in sensitive environments because it could introduce certain security vulnerabilities. See here for details.


This will interrupt your process (unless you start it in a thread), but you can use the code module to start a Python console:

import code
code.interact()

This will block until the user exits the interactive console by executing exit().

The code module is available in at least Python v2.6, probably others.

I tend to use this approach in combination with signals for my Linux work (for Windows, see below). I slap this at the top of my Python scripts:

import code
import signal
signal.signal(signal.SIGUSR2, lambda sig, frame: code.interact())

And then trigger it from a shell with kill -SIGUSR2 <PID>, where <PID> is the process ID. The process then stops whatever it is doing and presents a console:

Python 2.6.2 (r262:71600, Oct  9 2009, 17:53:52)
[GCC 3.4.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>

Generally from there I'll load the server-side component of a remote debugger like the excellent WinPDB.

Windows is not a POSIX-compliant OS, and so does not provide the same signals as Linux. However, Python v2.2 and above expose a Windows-specific signal SIGBREAK (triggered by pressing CTRL+Pause/Break). This does not interfere with normal CTRL+C (SIGINT) operation, and so is a handy alternative.

Therefore a portable, but slightly ugly, version of the above is:

import code
import signal
signal.signal(
        vars(signal).get("SIGBREAK") or vars(signal).get("SIGUSR2"),
        lambda sig, frame: code.interact()
        )

Advantages of this approach:

  • No external modules (all standard Python stuff)
  • Barely consumes any resources until triggered (2x import)

Here's the code I use in my production environment which will load the server-side of WinPDB (if available) and fall back to opening a Python console.

# Break into a Python console upon SIGUSR1 (Linux) or SIGBREAK (Windows:
# CTRL+Pause/Break).  To be included in all production code, just in case.
def debug_signal_handler(signal, frame):
    del signal
    del frame

    try:
        import rpdb2
        print
        print
        print "Starting embedded RPDB2 debugger. Password is 'foobar'"
        print
        print
        rpdb2.start_embedded_debugger("foobar", True, True)
        rpdb2.setbreak(depth=1)
        return
    except StandardError:
        pass

    try:
        import code
        code.interact()
    except StandardError as ex:
        print "%r, returning to normal program flow" % ex

import signal
try:
    signal.signal(
            vars(signal).get("SIGBREAK") or vars(signal).get("SIGUSR1"),
            debug_signal_handler
            )
except ValueError:
    # Typically: ValueError: signal only works in main thread
    pass

Tags:

Python

Recent Posts