Where should the bootloader be installed when I want to run Ubuntu from an external HDD?

First, it's CRITICAL that you understand your boot mode. Modern computers can boot in either of two ways:

• BIOS -- This is the old way of booting, which was used on most computers introduced prior to 2011. Modern computers support BIOS-mode booting via a feature called the Compatibility Support Module (CSM), aka "legacy-mode boot support" or a similar term. Thus, this method can be referred to by three names: BIOS, CSM, or legacy. It involves 16-bit boot loader code splatted in the Master Boot Record (MBR) and various other places. It's an inelegant but mature method of booting, and most sites that don't explicitly mention the boot mode describe BIOS-mode booting. If your computer came with Windows 8 or later, though, chances are it does not use this method of booting.
• EFI/UEFI -- This is the native boot mode for most computers introduced since mid-2011, and it is completely different from BIOS-mode booting, so you should ignore any advice that explicitly states it's for BIOS-mode booting, or anything that doesn't mention the boot mode (since it's likely to be old or written by somebody who doesn't know the difference). One exception: Confusingly, many people, and even computer manufacturers, refer to their EFIs as "BIOSes." This practice creates a great deal of confusion.

Since you mention Windows 10, chances are your computer is EFI-based and boots in EFI mode. I'll proceed under that assumption. If this is incorrect, ignore the rest of this answer.

To boot an internal disk in EFI mode, the OS typically stores a boot loader as a file on the EFI System Partition (ESP) on the disk. This filename can be anything legal, but for Ubuntu it will be EFI/ubuntu/shimx64.efi, which in turn launches EFI/ubuntu/grubx64.efi. The OS installer registers the name of the boot loader with the EFI's built-in boot manager, which stores that name in NVRAM. If the EFI works correctly (not all do), this results in a working boot process.

The trouble with this process, as applied to a removable disk, is three-fold:

• The Ubuntu installer will install GRUB to "the" ESP -- but you may have multiple ESPs available (almost certainly one on the hard disk and maybe one on the USB drive). Controlling where GRUB installs can be tricky. It is not done by using the option marked with the obvious title about where to install GRUB. Instead, it can be done (AFAIK) only from the "Something Else" installation option by marking the ESP you want to use as an "EFI boot partition" or "EFI System Partition" (which name depends on the Ubuntu version).
• If you install GRUB to the ESP on the USB drive, then when you remove the USB drive, the EFI may notice that it's got a registered boot loader entry that's not valid, and delete that entry. Many EFIs do this, and it means that if you install GRUB to an external disk and then unplug it, you lose the ability to boot from that disk, even when you plug the disk back in.
• The way Ubuntu configures GRUB causes it to rely on configuration files that are in the Ubuntu /boot/grub directory, which is not likely to be on the ESP. Thus, if Ubuntu installs GRUB to the ESP on your internal disk, you'll avoid the first problem but run into another one because when you start up with the USB drive disconnected, GRUB will launch and be unable to find its configuration files. The result is a grub> prompt that will mystify most users.

There are a number of solutions to these problems, including:

• Manual install of GRUB to the fallback filename on the external disk -- OS installers boot by using a fallback filename of EFI/BOOT/bootx64.efi. If you copy Shim (which launches GRUB) to this filename (by copying EFI/ubuntu to EFI/BOOT on the ESP and then renaming shimx64.efi to bootx64.efi in EFI/BOOT), your firmware will be able to launch your USB-based GRUB just as it did the Ubuntu installer. This will work past both of the problems I've just identified, but you'll need to use the computer's built-in boot manager to select which OS to boot. This built-in boot manager ranges from unusable to tolerable, so this might or might not be an acceptable solution. You may also need to tweak your boot order after you install Ubuntu to keep it from launching GRUB from either the internal disk's ESP or the external disk's ESP by default.
• Put a /boot partition on your internal disk -- If you install GRUB to the internal disk, then you can also put a /boot partition on the internal disk. This will enable GRUB to work normally on all boots; however, the Ubuntu option will appear all the time, even when the USB disk is unplugged. This may be undesirable, especially if Ubuntu is the default. Also, this solution means that your external USB drive will be bootable only on the computer on which you prepared it. If you want to be able to use that disk on other computers, you should avoid this solution.
• Use a smarter boot manager -- A boot manager gives you a choice of OSes to boot, whereas a boot loader loads a kernel into memory. GRUB does both jobs and the EFI's built-in boot manager does only the first job. There are other boot managers available, some of which may help you avoid the problems I've identified. My own rEFInd, for instance, if installed to the internal disk's ESP and made the default boot program, will detect and present an Ubuntu option if and only if GRUB or a Linux kernel is present -- that is, when the USB drive is plugged in. By setting the default_selection option in refind.conf appropriately, you can configure rEFInd to boot Ubuntu automatically if your Ubuntu USB drive is plugged in and to boot straight to Windows if the Ubuntu USB drive is not plugged in. My answer to this question provides more details about this type of configuration, albeit on a Mac rather than a PC.

With any luck, one of these approaches will get you up and running.