Upgrade unlocked package fails due to "Cannot use unfiled-public or user folders in a package"

I don't have an easy way to determine this immediately, but it may be due to the default upgrade type of Mixed. Try using the Delete mode instead:

sfdx force:package:install -p the_package_to_install -t Delete -u myAlias -w 10

I have managed to solve it. An email template with the exact same name has been created in the Sandbox in unfiled public, but wasn't part of the unlocked package. After deleting it, I could install the package.