What are the "things to know" when diving into multi-threaded programming in C++

I would focus on design the thing as much as partitioned as possible so you have the minimal amount of shared things across threads. If you make sure you don't have statics and other resources shared among threads (other than those that you would be sharing if you designed this with processes instead of threads) you would be fine.

Therefore, while yes, you have to have in mind concepts like locks, semaphores, etc, the best way to tackle this is to try to avoid them.

I am no expert at all in this subject. Just some rule of thumb:

1. Design for simplicity, bugs really are hard to find in concurrent code even in the simplest examples.
2. C++ offers you a very elegant paradigm to manage resources(mutex, semaphore,...): RAII. I observed that it is much easier to work with boost::thread than to work with POSIX threads.
3. Build your code as thread-safe. If you don't do so, your program could behave strangely

I am exactly in this situation: I wrote a library with a global lock (many threads, but only one running at a time in the library) and am refactoring it to support concurrency.

I have read books on the subject but what I learned stands in a few points:

1. think parallel: imagine a crowd passing through the code. What happens when a method is called while already in action ?
2. think shared: imagine many people trying to read and alter shared resources at the same time.
3. design: avoid the problems that points 1 and 2 can raise.
4. never think you can ignore edge cases, they will bite you hard.

Since you cannot proof-test a concurrent design (because thread execution interleaving is not reproducible), you have to ensure that your design is robust by carefully analyzing the code paths and documenting how the code is supposed to be used.

Once you understand how and where you should bottleneck your code, you can read the documentation on the tools used for this job:

1. Mutex (exclusive access to a resource)
2. Scoped Locks (good pattern to lock/unlock a Mutex)
3. Semaphores (passing information between threads)
4. ReadWrite Mutex (many readers, exclusive access on write)
5. Signals (how to 'kill' a thread or send it an interrupt signal, how to catch these)
6. Parallel design patterns: boss/worker, producer/consumer, etc (see schmidt)
7. platform specific tools: openMP, C blocks, etc

Good luck ! Concurrency is fun, just take your time...