Profiling a long-running Python Server

yappi profiler can be started and stopped at runtime.

There are two interesting tools that came up that try to solve that specific problem, where you might not necessarily have instrumented profiling in your code in advance but want to profile production code in a pinch.

• pyflame will attach to an existing process using the ptrace(2) syscall and create "flame graphs" of the process. It's written in Python.

• py-spy works by reading the process memory instead and figuring out the Python call stack. It also provides a flame graph but also a "top-like" interface to show which function is taking the most time. It's written in Rust and Python.

Not a very Pythonic answer, but maybe straceing the process gives some insight (assuming you are on a Linux or similar).

Using strictly Python, for such things I'm using tracing all calls, storing their results in a ringbuffer and use a signal (maybe you could do that via your HTTP message) to dump that ringbuffer. Of course, tracing slows down everything, but in your scenario you could switch on the tracing by an HTTP message as well, so it will only be enabled when your trouble is active as well.