headless ubuntu server with desktop gui with working gpu

I fixed it.

I supplied a EDID file from https://github.com/linuxhw/EDID/tree/master/Digital/Nvidia I configured Xorg using this nvidia-xconfig line:

sudo nvidia-xconfig -a --allow-empty-initial-configuration --use-display-device=None \
--virtual=1920x1200 --busid {busid} --use-display-device="DFP-0" \ 
--connected-monitor="DFP-0" --custom-edid="DFP-0:/etc/X11/edid.txt"

Where busid = gpu bus id

After that simply start x sudo startx and login.

later on I found that TurboVNC works a bit more stable. I installed VirtualGL and TurboVNC

made sure to forward the gpu to the VNC session and ran this:

sudo dpkg -i virtualgl_*.deb
sudo -i
/opt/VirtualGL/bin/vglserver_config

rmmod nvidia-nomode
rmmod nvidia-nvm
rmmod nvidia
reboot
xauth merge /etc/opt/VirtualGL/vgl_xauth_key
sudo dpkg -i turbovnc_*.deb
/opt/TurboVNC/bin/vncserver -geometry 1920x1080

Final xorg.conf