How to remove quarantined virus securely?

Although a lot depends on the anti-virus software, I'll try to answer your questions:

1) What is the best practice and secure way to deal with them? Delete from quarantine, or let them in there?

Do you plan to study the virus and its working? If yes, then you might want to keep it (This will require a restore of the file).

Do you think it might be a false positive? If yes, report it to your anti-virus vendor for a re-evaluation, and if it turns out to be a false positive, you can restore the file(s).

If you don't plan to do anything with the file (and don't want to restore it either), you can delete it.

2) If i delete a file from the quarantine, does it mean that it goes to the "recycle bin" of the computer and is thus still existent on my PC, and could be recovered by some tools like Recuva? Or will it be gone for ever

This largely depends on the implementation of the anti-virus. No reputable vendor would send the file to recycle bin.

Can they be recovered? Potentially yes. Even if the anti-virus program performs several iterations of deletion on that part of disk, the file could be potentially recovered using specialised techniques and tools.

3) What happens if there are files in the quarantine and you accidentally de-install the antivirus program from your PC, what happens to the files?

It should not affect your system, as the files are stored in a binary format. More details can be found here.

EDIT: As user user21820 points out in the comments, there's nothing to worry about if the file in the quarantine is deleted using multiple overwrites.

A computer virus is just a file, is not something that will be magically activated by itself and wreak havoc around. You can treat it just like a normal file: delete it and it's gone.

If you uninstall the AV, some will empty the quarantine directory, some not. In the event of the AV not clearing the quarantine, you will end up with a folder containing the virus file, so if you go there and delete the folder (or the file), the virus is gone.

Can it be recovered? Yes, it can. But not by itself. And it will not recover itself and activate itself.