How can I make a backup of an EFI System partition?

You can just copy everything inside it (i.e. e.g. sudo cp -R /boot/efi /path/to/backup is fine, however personally I'd suggest to use tar: sudo tar cfz /path/to/backup/ESP_backup.tar.gz /boot/efi);

The filesystem in which to store the backup is irrelevant; the only concern might be the permissions, but the UEFI firmware doesn't cater for Linux permissions (in fact FAT32 doesn't even support them), so that's not a concern at all.

Kos's answer is a good one. I want to bring up one more point, though: If you back up the ESP to a FAT partition on a USB flash drive, you may be able to boot using the backup in an emergency. That is, suppose that the ESP is accidentally trashed without your knowledge and you shut down the computer. When you power it up, the computer will no longer boot. If you plug in your backup, though, you may be able to boot from it, without using any other emergency recovery tools, and then copy the backup back to the ESP. For this to work, though, you'll need one of two things:

• A fallback boot loader -- The filename EFI/BOOT/bootx64.efi (or equivalent for other architectures) on a FAT partition is the fallback filename, which the firmware uses if no other NVRAM-specified option works, or when booting from removable media. Some OSes (like Windows) automatically put a copy of their boot loaders under this name, in addition to their regular name. Ubuntu doesn't do this, but you can create such a copy easily enough -- just copy the EFI/ubuntu directory (on the ESP or on your backup) to EFI/BOOT and then rename shimx64.efi (or grubx64.efi, if shimx64.efi is absent) in the EFI/BOOT directory to bootx64.efi.
• Firmware support for booting from an arbitrary file -- Some EFIs provide a way for you to boot from an arbitrary file. You pick a "boot from file" option in the firmware setup utility or boot manager, whereupon the EFI shows you a file-selection tool. You use that tool to browse to the boot loader you want to use -- EFI/ubuntu/shimx64.efi (for Secure Boot support) or EFI/ubuntu/grubx64.efi in this case.

You can set up your backup with a fallback boot loader filename yourself, so the first approach will work on any EFI-based computer. The second depends on the features provided by the manufacturer, though, so you should check to see if this feature is available before you rely on it. Either way, testing your ability to boot in this way may be worth doing.

If your backup is in some other form (like a tarball or buried deep in a directory tree) or on a filesystem other than FAT, and if the ESP is damaged to the point where you can't boot, you'll need to use an emergency system (an Ubuntu "live CD," a rEFInd image on removable disk, etc.) to boot either an emergency system or your regular installation, from which you can restore the backup. This isn't a huge hassle, of course, although if you don't have such a tool handy already it could be a nuisance, especially if you don't have another computer on which to download and prepare the recovery tool. Keeping the backup in the form of a removable disk might therefore make things go just a little bit more smoothly.