How to connect to VirtualBox remote desktop client using VNC?

VNC doesn't work in this manner because VirtualBox doesn't support RFB; it supports RDP instead. RFB and RDP are entirely different protocols that are not compatible.

• RFB is VNC's open Remote Frame Buffer protocol -- it uses TCP port 5900 by default, and also serves HTTP over TCP port 5800 to support a convenient client applet written in Java

• RDP is Microsoft's proprietary Remote Desktop Protocol -- it uses TCP port 3389 by default

You'll either need to use some client software that supports RDP sessions, and/or install a VNC server in Windows XP (and then your Chicken of the VNC client will be able to connect), to get remote access to your virtual Windows XP guest environment.

I highly recommend the famously popular UltraVNC because I've had good experiences with it as a fast and reliable solution in all editions of Windows as far back as Windows XP:

  UltraVNC (free and open source)
  http://www.uvnc.com/

Quoting the Virtual Box manual:

http://www.virtualbox.org/manual/ch09.html#otherextpacks

Starting with VirtualBox 4.2.0 there is another extension pack, VNC, which is open source and replaces the previous integration of the VNC remote access protocol. This is experimental code, and will be initially available in the VirtualBox source code package only. It is to a large portion code contributed by users, and is not supported in any way by Oracle.

The keyboard handling is severely limited, and only the US keyboard layout works. Other keyboard layouts will have at least some keys which produce the wrong results (often quite surprising effects), and for layouts which have significant differences to the US keyboard layout it is most likely unusable.

It is possible to install both the Oracle VM VirtualBox Extension Pack and VNC, but only one VRDE module can be active at any time. The following command switches to the VNC VRDE module in VNC:

VBoxManage setproperty vrdeextpack VNC

Configuring the remote access works very similarly to VRDP (see Section 7.1, “Remote display (VRDP support)”), with some limitations: VNC does not support specifying several port numbers, and the authentication is done differently. VNC can only deal with password authentication, and there is no option to use password hashes. This leaves no other choice than having a clear-text password in the VM configuration, which can be set with the following command:

VBoxManage modifyvm "VM name" --vrdeproperty VNCPassword=secret

The user is responsible for keeping this password secret, and it should be removed when a VM configuration is passed to another person, for whatever purpose. Some VNC servers claim to have "encrypted" passwords in the configuration. This is not true encryption, it is only concealing the passwords, which is exactly as secure as clear-text passwords.

The following command switches back to VRDP (if installed):

VBoxManage setproperty vrdeextpack "Oracle VM VirtualBox Extension Pack"

Your best options are:

1. Find an RDP viewer for Mac (or wait for VB's included rdesktop-vrdp client to become available for Mac). See this for instructions on how to set up authentication:

2. Use VNC instead of RDP as VB's virtual remote desktop plugin.