Multiprocessing Errors - ArcGIS implementation

Each IWorkspace connection (i.e each database connection) has thread affinity. Two threads cannot share the same workspace. You can have one thread own the resource and then sync the access, but if you are going to be using straight gp functions, then that is even not an option.

The easiest (lame) way is to create separate processes and then do multi process synchronization (as opposed to multithread synchronization). Even then you should be aware of the underlying workspace type. if you are not using arcsde (a multi-user datasource) you will probably use a single user datasource (like personal or filegdb). Then remember that means only one process can write at a time! The typical (lame) synchronization for these scenarios is that each parallel process writes to a different temp workspace and then you merge it all in to your destination workspace in a single process.

You have several threads competing for the same resource.

Try moving your 'import arcpy' statement into the target of the multiprocessing. You'll ensure arcpy is working with it's own set of environment variables and memory.

It sounds absurd, but even though you are setting environment variables in the Multiprocess target method, python is still using a shared memory space to manage the arcpy module and therefore any variables you set.

Arcpy isn't thread safe. It was always intended to be used within a single process. But there are workarounds.

---

My suggestion was to import arcpy within the target for new process.

def _multiprocessing_target(args):
    import arcpy
    ...code