Execute arbitrary python code remotely - can it be done?

Take a look at PyRO (Python Remote objects) It has the ability to set up services on all the computers in your cluster, and invoke them directly, or indirectly through a name server and a publish-subscribe mechanism.


It sounds like you want to do the following.

  • Define a shared filesystem space.

  • Put ALL your python source in this shared filesystem space.

  • Define simple agents or servers that will "execfile" a block of code.

  • Your client then contacts the agent (REST protocol with POST methods works well for
    this) with the block of code. The agent saves the block of code and does an execfile on that block of code.

Since all agents share a common filesystem, they all have the same Python library structure.

We do with with a simple WSGI application we call "batch server". We have RESTful protocol for creating and checking on remote requests.


Stackless had ability to pickle and unpickle running code. Unfortunately current implementation doesn't support this feature.

Tags:

Python

Grid