How to start a math blog?

Wordpress is a fairly straightforward answer. There are already many math blogs on it, e.g. Terence Tao's and Tim Gowers'. Requires minimal setup.

If you don't want such a complex panel or want to start simple.

Just make an account on "http://blogger.com" or "https://www.tumblr.com/". They are very straight forward. And later you can move on to a professionally managed platform if you start getting a lot of traffic.