How do I remove automatically added keyboard layouts?

I was able to remove the unwanted layouts by creating an IgnoreRemoteKeyboardLayout DWORD in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout and setting it to 1

Credit goes to this post.

Add the offending languages under language in control panel, then delete them and they will be gone from the notification area.

The keyboard layout (in Windows 7 at least) is contained in the registry key HKEY_USERS\.DEFAULT\Keyboard Layout\Preload.

Preload determines which keyboard layouts are present in the keyboard switch button (on the bottom right of the taskbar), and the number determines the ordering. The REG_SZ item "1" contains the first layout, "2" the second, etc.

The article Windows Keyboard Layout lists most keyboards ids. You would also find your above layout ids defined as sub-keys of the registry key :
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\keyboard layouts.

It would be interesting to know what your registry contains at these locations.