Why did they design new connectors for USB?

Most likely many reasons, at least the following.

It was a requirement for the connector to support hot-plugging. The connectors you mentioned do not support hot-plugging.

Using an existing connector also means it is possible for someone to plug in two incompatible devices together just because they use the same connector.

The connector also needs to support enough mating cycles to be usable.

And the connectors you mention are old, as technology to make connectors advances, it makes sense to use that technology to make a modern connector that is smaller and more durable and otherwise better, and most importantly, a connector that is suitable for the specific application.

Compatibility with existing connectors is an anti-feature

Let's imagine they used the then-common DB-9 DE-9 for USB. What would happen? Lots of mice used a DE-9, and it's used in a lot of other serial ports. People would be plugging a serial mouse into the USB port, and become frustrated when it didn't work.

Availability of USB connectors is a problem that would solve itself

Of course, the hardware person wants to see pre-existing connectors so they could just buy them at Digi-Key. However, this problem would solve itself fairly soon, as manufacturers started offering the new design.

They Learned from Parallel & Serial Ports

Until the IBM-PC came along, the "typical" parallel port was the 36-pin "Centronics" connector. Then IBM decided to use a DB-25F connector for the parallel port on the IBM PC. Oops, that was already very commonly used for serial ports on computers and terminals. So they switched the serial port to a DB-25M connector, which meant new cables or gender changers were needed if you put an IBM-PC in place of a terminal, even if the pinout was the same.

So with USB, they created something new, and that was a very good idea.