What is the difference between HTTP/1.1 pipelining and HTTP/2 multiplexing?

HTTP/1.1 without pipelining: Each HTTP request over the TCP connection must be responded to before the next request can be made.

HTTP/1.1 with pipelining: Each HTTP request over the TCP connection may be made immediately without waiting for the previous request's response to return. The responses will come back in the same order.

HTTP/2 multiplexing: Each HTTP request over the TCP connection may be made immediately without waiting for the previous response to come back. The responses may come back in any order.

HTTP/1.1 pipelining still requires the requests to be returned in full, in the order requested.

HTTP/2 allows the requests responses to be split into chunks and be returned in an intermingled fashion so avoiding head of line blocking.

Additionally HTTP/1.1 pipelining never really took off and browser and server support is limited (see: https://en.m.wikipedia.org/wiki/HTTP_pipelining).

But yes, in theory, they are similar and hence give similar performance benefits. HTTP/2 is just a better, more fully featured, more supported version of this - along with other benefits that you've noted.

See also my answer here for a deeper discussion on HTTP/2 multiplexing: What does multiplexing mean in HTTP/2