Is there a way to attach a debugger to a multi-threaded Python process?

Use Winpdb. It is a platform independent graphical GPL Python debugger with support for remote debugging over a network, multiple threads, namespace modification, embedded debugging, encrypted communication and is up to 20 times faster than pdb.

Features:

  • GPL license. Winpdb is Free Software.
  • Compatible with CPython 2.3 through 2.6 and Python 3000
  • Compatible with wxPython 2.6 through 2.8
  • Platform independent, and tested on Ubuntu Gutsy and Windows XP.
  • User Interfaces: rpdb2 is console based, while winpdb requires wxPython 2.6 or later.

Screenshot
(source: winpdb.org)


Yeah, gdb is good for lower level debugging.

You can change threads with the thread command.

e.g

(gdb) thr 2
[Switching to thread 2 (process 6159 thread 0x3f1b)]
(gdb) backtrace
....

You could also check out Python specific debuggers like Winpdb, or pydb. Both platform independent.


You can attach a debugger to a multi-threaded Python process, but you need to do it at the C level. To make sense of what's going on, you need the Python interpreter to be compiled with symbols. If you don't have one, you need to download source from python.org and build it yourself:

./configure --prefix=/usr/local/pydbg
make OPT=-g
sudo make install
sudo ln -s /usr/local/pydbg/bin/python /usr/local/bin/dbgpy

Make sure your workload is running on that version of the interpreter. You can then attach to it with GDB at any time. The Python folks have included a sample ".gdbinit" in their Misc directory, which has some useful macros. However it's broken for multi-threaded debugging (!). You need to replace lines like this

while $pc < Py_Main || $pc > Py_GetArgcArgv

with the following:

while ($pc < Py_Main || $pc > Py_GetArgcArgv) && ($pc < t_bootstrap || $pc > thread_PyThread_start_new_thread)

Otherwise commands like pystack won't terminate on threads other than the main thread. With this stuff in place, you can do stuff like

gdb> attach <PID>
gdb> info threads
gdb> thread <N>
gdb> bt
gdb> pystack
gdb> detach

and see what's going on. Kind of.

You can parse what the objects are with the "pyo" macro. Chris has some examples on his blog.

Good luck.

(Shoutout for Dan's blog for some key information for me, notably the threading fix!)

Tags:

Python

Debugging

Recent Posts