Why not make the second factor the only factor?

I think there are two independent questions here that you need to distangle.

Question #1: Should I use just one factor or should I use two?

This is the question that you adress in your title - "Why not make the second factor the only factor?".

Two factor will be safer than one, but it comes with a usability and an implementation cost. If it is worth the cost depends on what your customers are prepared to accept and how valuable the thing you are trying to protect is. For a bank 2FA is almost a must, for an online hobby forum is is probably overkill. No general one size fits all answer is possible.

Question #2: If I use just one factor, which one should you pick?

This is the question the link to Medium (mostly) adresses. It is not the same as the first question.

This basically boils down to "what kind of authentications is best" - obviously a very broad question without a definative answer. But lets look at the scheme you mention.

Using email instead of a password? Can probably be a good idea for a low value target. My hobby forum on shared hosting is way more likely to be breached than the GMail. But if I am a bank this is probably bad, since some customers might have very weak email providers.

Using smartphone instead of a password? If your primary threat is large bot brute force attacks or reused credentials from other breaches this is probably a good idea. If your primary threat is snooping from someone close to you this is probably bad - there are probably a few people in your life that has easy access to your smartphone. So for the hobby forum I would say yes, but for Ashley Madison maybe no. (Or on second thought, maybe it would have been the best for them...)

I think the take home lesson here is that the one factor does not have to be a password, but what it should be depends on the specific circumstances of your situation.