Why does anti-virus software not delete the viruses, malware, etc., but instead quarantine them?

Viruses and malwares are not dangerous if not executed.
A file in quarantine cannot be executed by the user and the malicious code (virus or malware) has no possibility to act. If the virus/malware is removable it will be removed immediately.
If not the file will be moved to quarantine.

There are different reasons for this:

• False positive (as stressed out by other answers as well, see below in Further explanation).
• Future possibility to recover the file (the virus adds its code inside the original file and move/crypt/hide part of the original code somewhere. At present it is not possible to recover the file but maybe in a near future it will be).
  Indeed if the file is unique (e.g. one created by the owner of the computer) and it is somehow precious, the user may find a way to recover all the parts that are still possible to recover from it. A part of a thesis (or of an image) is always better than nothing.

• Possibility to study the virus by the antivirus company or to individuate other computer with the infection (let's imagine you have a file attacked by a virus. Its signature, md5sum changes. You have the same file on many computers. If the signature is the same you can guess they are attacked. If you check in your backups you can find the first time the virus acted).
  _{Note: historically the "quarantena" was a period of 40 days isolation for ships and people before entering the city in order to prevent the diffusion of the Black Death, to see if the virus develops or not. On our computers the quarantine is just a safe place were to keep inactive the suspect files, without observing any actions of the virus.}

• In the quarantine can end up even an executable file that is changed.
  Imagine that you have a program that you recompile or an open source program that is updated not via usual windows ways: the antivirus can notice activities (writing) on an exe-cutable file and put it in quarantine.
  Moreover since there are some files with active content (as, e.g., Word or eXcel macro...) some antivirus can spot differences in the executable parts and interpret those as produced by the action of a virus.

• If you have the same version of a file attacked from a virus in different ways, it can be (theoretically) possible to recover the file by crossing and analysing data of these versions.

Further explanation
Think like a virus and an antivirus to understand why the quarantine exists, why there can be false positives and why this is a battle that continues each day.

A virus (or a malware) is a compiled code that executes the purpose for what was programmed for.
As compiled code, it's binary (usually) and not text (as what you are reading). It has to propagate itself and to execute some homework (a mission, technically a payload), not necessarily at the same time (this increases the possibility of spreading the infection before it is detected).

How can a virus propagate itself and be executed?

• Simply it can overwrite a part of the original code (exe,dll,com... files) and put its code instead.

  
  ^{Example of an ancient DOS virus that acts in a such mode.}
  The drawback is that the original program can stop working and the virus may be detected faster (E.g.: "...hello my program is not working... strange things are happening... can you help? - Yes sir you have a virus").

• It can copy the initial part of file to be infected at its end, after it can put itself instead of the first part. So when you execute the program, the virus is firstly executed and only then the program is executed... A smarter variant is to copy itself at the end of the file and to put a jump to the end in the beginning of the file (and one back to its beginning at its end)... The drawback is that an antivirus can search for the code of the virus (once known) and find it easily. This happened in the Cascade virus in the 80s-90s...

  

• It can be made of parts and he (note not it) can change his shape and hide himself in different parts of the program, move them, encrypt and scramble. Each time he may infect a new file in a different way. Therefore the antivirus may only find remains in fingerprints – each day he is harder to identify.

Now, do you remember that the virus is (usually) binary code? Well, the fingerprints are too.
Since they are not the full virus but only a few bytes, it may happen that a part of a compressed file, data file, or image has the same bytes of one of the many known viruses fingerprints – hence the false positive.

Conclusive note: not all the viruses were planned to damage, but most of them do it, de facto.
With the actual use of computers with bank accounts and bills to pay, it doesn't seem any more as funny as the images above.

Anti-malware applications provide a quarantine option, which is often on by default in order for two reasons:

1. Keep a backup of the items identified as threatening in case of a false positive. Although not very common, I have seen cases of false positive on many different legitimate application files and drivers.
2. Having the item in quarantine may allow it to be better investigated. The fact that it matches a malware signature doesn't mean that it is just similar but may actually have other particularities.

For the same reason that (most) governments arrest suspected criminals instead of shooting them on the street at the slightest provocation:

You want to give the suspect a chance to defend themselves, in case they actually did not commit any crime at all. And, even if they did commit a crime, you probably want to find out all about it.