How to best clone a running system to a new harddisk using rsync?

You can grab the UUID for all block devices with the blkid command. (You want the one that just says UUID, not PARTUUID)

The rsync options I use are -avhPHAXx.

I don't think -v or --progress will speed anything up unless you're on a very slow console/tty.

Using -x eliminates the need for all of your exclusions assuming they are all on different filesystems (on my system, all except lost+found are).

The only frequently used program that I know of that uses hard links (at least on my system) is git, so that's why I add the -H option. The only problem I think you would have by not using -H is that it will take up a little more space.

As for the bootloader, if you are using GRUB2 with MBR, then the command I use is grub-install /dev/sda (replace sda with the correct drive for you). That should make the new drive bootable. If you're using a different bootloader or UEFI, then I would check google for how to get the new drive booting properly. Just remember that /boot on the new drive will need to be on the same partition as it currently is (assuming you're not using a UUID for /boot also), otherwise you will need to modify fstab accordingly.

I just did this successfully (after a couple trys).

I used

sudo rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /mnt

Then I reset my /mnt/etc/fstab file for the boot partition and my swap space.

Then I needed to reset GRUB

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
sudo grub-install --recheck /dev/sdX
sudo update-grub

Do not attempt to fix booting with this drive; my recommendation:

1. Plug in a drive on another machine instead of risking a hotswap on the failing machine.
2. rsync your non-system files across to the new disk.
3. On a separate hard disk, create a boot, a minimal root, and a swap. Install the same operating system that you wish to clone from the ailing hard disk.
4. Boot from this new disk (ideally on a separate machine, if you cannot risk downtime on the target host, otherwise you can boot the target host with this disk).
5. Add the disk from step #2 to this new system, with the correct mount points. Now you have a clone of the system drive. You can (optionally) copy the partitions over to this new disk, but my recommendation is to keep the disk as a minimum requirement for booting your OS. As you have realized, having too many things on the primary partition makes recovery difficult. Make sure you copy the network configuration correctly as your server is headless.
6. Simply replace the drive on the ailing system with this new disk pair (if you choose not to boot the target initially in step #4).
7. Reboot.