Why is MySQL connecting through socket even though I'm specifying host and port?

You need to use 127.0.0.1 instead of localhost:

> mysql -h 127.0.0.1 -P 3306
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8347701
Server version: 5.6.35-80.0-log Percona Server (GPL), Release 80.0, 
Revision f113994f31
...

You're in!