Apple Developer - What is the 'Fonts' Capability under Register an App ID for?

Since iOS 13, apps can make use of a Fonts entitlement, with a sub option 'Use Installed Fonts' to use custom fonts installed by special font provider apps. Font provider apps make use of the same entitlement, but with sub option 'Install Fonts'. Examples of such apps are CreativeCloud from Adobe and MyFonts from MonoType.

You only need to add the Fonts entitlement to your app, if you want to use fonts installed by said font provider apps, or if you are developing a font provider app yourself.

Does anyone know in what circumstances do you need to tick that Fonts capabilities box and how to use it?

Yes, new in iOS 13 your app can include fonts and can make them available to all the user’s apps by calling CTFontManagerRegisterFontURLs. This is called installing fonts. You need the capability to do that. The capability is explained at 5:46 of the video you cited in your question. Watch the video, it explains all about this.

You do not need the entitlement merely to include and use fonts within your own app.