Why using the premaster secret directly would be vulnerable to replay attack?

I think you misunderstand what a replay attack is. You are correct that a MitM wouldn't be able to see the premaster secret, but an attacker can still cause trouble even if they can't decrypt anything.

Say you make a request to the server ordering a widget. There's a MitM between you and the server, but the request is perfectly secure, and they can't decrypt anything. Unfortunately, this is an alternate universe where the premaster secret is used directly.

The MitM sees you've made a request to store.example.com. They can't see what the request is, but they can see that it's to store.example.com (either due to the IP address or due to SNI). They can't see any of the decrypted communications, but they can see all of the encrypted traffic. Just for the fun of it, they take all the encrypted traffic you sent to the server, and they send it again, 10 times.

Now your card is charged 10 more times and a few days later you end up with 11 widgets instead of 1.

---

Now what would happen if an attacker tried that in our universe? (very simplified)

1. The attacker sends a copy of the victim's traffic that it had saved
2. In the Server Hello, the server replies with a different ServerHello.random
3. All values the attacker sends that are based on the original ServerHello.random won't match
4. When the server receives the Finished message and tries to verify it, it will fail, because the master secret calculated by the server is different (even though the premaster secret is the same)
5. The server will abort the connection

The key point here is that there is some state on the server, the ServerHello.random, that is different for each connection, and the master secret depends on this state.