How do I create the small icon next to the website tab for my site?

It is called favicon.ico and you can generate it from this site.

http://www.favicon.cc/

This is for the icon in the browser (most of the sites omit the type):

<link rel="icon" type="image/vnd.microsoft.icon"
     href="http://example.com/favicon.ico" />

<link rel="icon" type="image/png"
     href="http://example.com/image.png" />

<link rel="apple-touch-icon"
     href="http://example.com//apple-touch-icon.png">

for the shortcut icon:

<link rel="shortcut icon"
     href="http://example.com/favicon.ico" />

Place them in the <head></head> section.

Edit may 2019 some additional examples from MDN

Related