How to start GUI from command line in Android-x86

To start the GUI, I had to select VboxVGA instead of VMSVGA in the VirtualBox display settings of the virtual machine.

Note: This is a solution to make it work with VirtualBox in case it helps someone who finds this question. It is possible that the solution to make it work with VMware player is similar.

I also found this information here:

Mandatory: Unless guest additions are installed, change the default VMSVGA to VBoxVGA.

To fix this, you have to open the Virtual Machine Settings for your Android virtual machine and then go to the Display settings on the Hardware tab. Make sure that 3D acceleration is ticked! Start your Android virtual machine and wait maybe up to 10 seconds on the command line. It will then boot to Android GUI.