Switching to a second X session kills the first

You have a kernel crash in the nouveau module, which will screw up the DRM module, which in turn will screw up an X server connecting to that module when it hits the bad state.

Upgrade to the newest kernel and matching nouveau module possible. If you still get the same crash, file a bug report with the nouveau developers. Make sure to include the full dmesg and Xorg.*.log.