What is the difference between DTR/DSR and RTS/CTS flow control?

The difference between them is that they use different pins. Seriously, that's it. The reason they both exist is that RTS/CTS wasn't supposed to ever be a flow control mechanism, originally; it was for half-duplex modems to coordinate who was sending and who was receiving. RTS and CTS got misused for flow control so often that it became standard.

• DTR - Data Terminal Ready
• DSR - Data Set Ready
• RTS - Request To Send
• CTS - Clear To Send

There are multiple ways of doing things because there were never any protocols built into the standards. You use whatever ad-hoc "standard" your equipment implements.

Just based on the names, RTS/CTS would seem to be a natural fit. However, it's backwards from the needs that developed over time. These signals were created at a time when a terminal would batch-send a screen full of data, but the receiver might not be ready, thus the need for flow control. Later the problem would be reversed, as the terminal couldn't keep up with data coming from the host, but the RTS/CTS signals go the wrong direction - the interface isn't orthogonal, and there's no corresponding signals going the other way. Equipment makers adapted as best they could, including using the DTR and DSR signals.

EDIT

To add a bit more detail, its a two level hierarchy so "officially" both must happen for communication to take place. The behavior is defined in the original CCITT (now ITU-T) standard V.28.

The DCE is a modem connecting between the terminal and telephone network. In the telephone network was another piece of equipment which split off to the data network, eg. X.25.

The modem has three states: Powered off, Ready (Data Set Ready is true), and connected (Data Carrier Detect)

The terminal can't do anything until the modem is connected.

When the terminal wants to send data, it raises RTS and the modem grants the request with CTS. The modem lowers CTS when its internal buffer is full.

So nostalgic!