Can a website know about my browsing history?

The short answer is yes, though it is not as easy as you might think.

The browser stores cookies independently for each domain. That means that www.foo.com cannot access the cookies made by www.bar.com and vice versa.

The vulnerability (or loophole) is in included pages. Most advertisements come from a different domain than the page itself, so they create their own sets of cookies. When another site includes an ad from the same ad provider, they can read their own cookies created earlier and they know you visited that page earlier. This way they can only track you on sites that host their ads. This is the strategy Google uses to serve relevant ads.

Also Facebook and other social networks can do this because of their ubiquitous like, tweeet, pin etc. buttons, which are also included content. This is not avoidable without disabling cookies altogether or using private browsing, but that could be a major burden (Cookies do make the Internet convenient). I personally choose not to be paranoid of these things.