Why do most browsers store browsing history by default?

There are a lot of advantages. Here are some:

1. Auto-completion of previously visited URLs you forgot, which can speed up the web surfing process tremendously. You might have remembered parts of a URL or website title, and your browser can usually pick those up if you typed them in. I love this feature.
   • This can offer extra security. As mentioned by kasperd, it can greatly reduce the risk of typosquatting.
2. Storing previously-loaded data in a cache to speed up web-browsing. Great for slow connections. Great for reducing load on web servers.
3. Storing cookies so websites remember your login information, etc.

The HTML standard is that unvisited links and previously-visited links are styled differently. By default in most browsers, an unvisited link is blue and a visited link is pink, but nearly every web page these days overrides that. In order to do this, you have to have a list of previously visited links so that you know what style to use for every link on every page, which is to say a browser history.

Because it fits the majority

Surveys (e.g. http://www.aleecia.com/authors-drafts/tprc-behav-AV.pdf) show that something like ~25% of people have ever used incognito mode and ~50% of people have ever cleared browser history.

Needless to say, many of them don't do that all the time, so at any given moment a majority of users prefer to use a mode that does store browsing history, thus it's reasonable to enable it by default.