What is the purpose of leading slash in HTML URLs?

It's important to start a URL with a / so that the intended URL is returned instead of its parent or child.

Let's say if your browsing /cream-cakes/ then you have a link on the page that has blah.html without the forward slash it is going to attempt to visit page /cream-cakes/blah.html while with the forward slash it'll assume you mean the top level which will be domain.com/blah.html.

Generally, it's best to always use / in my experience as its more friendly when you change the structure of your site, though there is no right or wrong assuming that the intended page gets returned.

Does the leading '/' mean the path is starting from the site root?

Technically this is referenced in section 4.2 of RFC 3986 as an "absolute-path reference":

A relative reference that begins with a single slash character is termed an absolute-path reference.

It ensures the path is absolute to the root directory and not the current directory (termed a "relative-path" reference). See this for an expanded discussion on that.

That's a root-relative link. It's a relative link (somewhat akin to ../) but it begins at the root of the site. If a page three levels deep on the site begins a link with the forward slash, the remainder of the path will be relative to the root of the site.

A benefit to this form of pathing is fewer characters in the markup:

http://example.com/page.html

vs

/page.html

Another advantage is portability across domain changes. If example.com content is moved to example.org, for example, root-relative links will still work, assuming the same directory naming/layout is used. Especially useful if developing pages locally, then uploading to the web.

As with other types of pathing - relative (../) and absolute (http://...) this is still subject to updating links when files or directories are renamed or moved.