Apple - What is standard for OS X filesystem? e.g. /opt/ vs. /usr/

Is there a manual or guide to what all of the built-in directories should be used for? Like what is /home/, or /net/?

Take a look at the File System Programming Guide for the most up-to-date information and at man hier in Terminal, which provides a "historical sketch" of the filesystem hierarchy (it's included at the end of this answer for reference).

A comment to your question mentions the Filesystem Hierarchy Standard. You will probably come to the conclusion, after perusing File System Programming Guide and man hier, that macOS doesn't follow the FHS, that's more of a Linux thing. Of course, there are similarities between the FHS and the filesystem layout in macOS because of the common UNIX origin, but the differences are striking. macOS doesn't use any of these:

  • /boot folder -> macOS uses /System/Library/Kernels instead (in older versions of macOS, the folder containing the kernel was /)
  • /home folder -> macOS uses /Users instead
  • /root folder -> macOS uses /var/root instead

and /opt isn't mentioned not one time in any document (more on /opt below.)

A further distiction between macOS and a FHS-compliant OS is the use of /private, for example /etc is a symlink to /private/etc.

About /net: It is an automounter map (listed in /etc/auto_master), see Wikipedia for more information.

What does and should go into /opt/ and what should/does go into /usr/? My understanding is that /usr/ used to be for user home directories, but since that exists in /Users/ - what is the purpose now?

Although /usr was used in the past to place the home directories of the users, that's no longer the case.

Nowadays, /usr contains user commands (in /usr/bin for normal users and /usr/sbin for administrative users, like root), shared libraries (/usr/lib), man pages (/usr/share/man), executables that shouldn't be run directly by users (/usr/libexec) and other stuff.

It also offers a subdirectory, /usr/local, to place programs, libraries and other files that don't come with the base OS.

/opt has a very similar role to /usr/local and they seem interchangeable. However, from my experience working with other Linux/UNIX sysadmins, there seems to be a preference for /usr/local in BSD-based UNIX OSs.

So this is my take on it: macOS is BSD-based and consequently I'd use /usr/local. Note that you can create a program directory and then symlink commands to /usr/local/bin, etc, for example:

/usr/local/mysql
/usr/local/mysql/bin/mysqladmin
/usr/local/mysql/lib/libmysqlclient.so
/usr/local/bin/mysqladmin -> ../mysql/bin/mysqladmin
/usr/local/lib/libmysqlclient.so -> ../mysql/lib/libmysqlclient.so

This used to be usual practice in Linux and UNIX too, but the FHS explicitely forbids it: if you wish to install third party packages in their own directory hierarchy you should use/opt/<package> instead. Note that FHS-compliance requires to put configuration files in /etc/opt/<package> and variable files in /var/opt/<package>.

So, in macOS, I'd recommend that you stick to /usr/local as described above.

I'm aware of add-on software like Cisco VPN and XQuartz that install in /opt, so the above distinctions start to blur.

man hier

As mentioned above this is man hier:

 A historical sketch of the filesystem hierarchy.  The modern macOS filesystem is documented in the
 ``File System Programming Guide'' available on Apple Developer.

 /             root directory of the filesystem

 /bin/         user utilities fundamental to both single-user and multi-user environments

 /dev/         block and character device files

               fd/  file descriptor files; see fd(4)

 /etc/         system configuration files and scripts

 /mach_kernel  kernel executable (the operating system loaded into memory at boot time).

 /sbin/        system programs and administration utilities fundamental to both single-user and multi-
               user environments

 /tmp/         temporary files

 /usr/         contains the majority of user utilities and applications

               bin/      common utilities, programming tools, and applications
               include/  standard C include files

                         arpa/       C include files for Internet service protocols
                         hfs/        C include files for HFS
                         machine/    machine specific C include files
                         net/        misc network C include files
                         netinet/    C include files for Internet standard protocols; see inet(4)
                         nfs/        C include files for NFS (Network File System)
                         objc/       C include files for Objective-C
                         protocols/  C include files for Berkeley service protocols
                         sys/        system C include files (kernel data structures)
                         ufs/        C include files for UFS

               lib/      archive libraries
               libexec/  system daemons & system utilities (executed by other programs)
               local/    executables, libraries, etc. not included by the basic operating system
               sbin/     system daemons & system utilities (executed by users)
               share/    architecture-independent data files

                         calendar/  a variety of pre-fab calendar files; see calendar(1)
                         dict/      word lists; see look(1)

                                    web2        words from Webster's 2nd International
                                    words       common words

                         man/       manual pages
                         misc/      misc system-wide ascii text files
                         mk/        templates for make; see make(1)
                         skel/      example . (dot) files for new accounts
                         tabset/    tab description files for a variety of terminals; used in the term-
                                    cap file; see termcap(5)
                         zoneinfo/  timezone configuration information; see tzfile(5)

 /var/         multi-purpose log, temporary, transient, and spool files

               at/        timed command scheduling files; see at(1)
               backups/   misc. backup files
               db/        misc. automatically generated system-specific database files
               log/       misc. system log files

               mail/      user mailbox files
               run/       system information files describing various info about system since it was
                          booted

                          utmpx       database of current users; see utmpx(5)

               rwho/      rwho data files; see rwhod(8), rwho(1), and ruptime(1)
               spool/     misc. printer and mail system spooling directories

                          mqueue/     undelivered mail queue; see sendmail(8)

               tmp/       temporary files that are kept between system reboots
               folders/   per-user temporary files and caches