C++: implementation of a class methods in a separated shared library

Short Answer

It won't work the way you are doing it. There might also be other problems with your approach that haven't bitten you yet.

Why it doesn't work

Undefined symbols in your program/library are resolved at different times. On most systems, data references (global variables, class vtables, etc.) are always resolved when your program/library is loaded. Code references are resolved when they are first used on some systems ("lazy lookup"; it happens on Linux and Mac OS X, at least), unless some special options are set (RTLD_NOW parameter for dlopen or LD_BIND_NOW environment variable). Once these are resolved, no new lookup will take place.

If you dlopen your library with the RTLD_GLOBAL flag before the lazy lookup for a method is done, the method from your library will be used. The code reference to the method is now resolved; it won't change again. Your main program now officially uses symbols from your dlopened library, so dlclose will no longer close the library - dlclose only drops your explicit handle to it.

In short, you should only ever expect to unload libraries that you only use via explicit calls to dlsym.

What to do instead

What you can do instead is have your library provide a derived class implementation. You'd define your class C as an abstract base class:

class C
{
public:
    virtual void method();
};

In your separately-compiled library, you'd define a derived class and a function that creates an object of that derived class:

class D : public C
{
public:
    virtual void method();
};

void D::method()
{
    // ...
}

extern "C" C* createC()
{
    return new D();
}

Now, in your main program, you'd load the library using dlopen, get a function pointer to createD using dlsym and call it to get an object. When all objects are gone, you can call dlclose, recompile your library, and do the whole thing again:

typedef C* (*creatorFunction)();

int main()
{
    for(;;)
    {
        void *handle = dlopen("mylib.so", 0);
        creatorFunction create = (creatorFunction) dlsym(handle, "createC");

        C *c = (*create)();
        c->method();
        delete c;

        dlclose(handle);

        char pause;
        cin << pause;
    }
    return 0;
}