What's the difference between the server version and the desktop version?

Copied as-is from Ubuntu docs:

• The first difference is in the CD contents. The "Server" CD avoids including what Ubuntu considers desktop packages (packages like X, Gnome or KDE), but does include server related packages (Apache2, Bind9 and so on). Using a Desktop CD with a minimal installation and installing, for example, apache2 from the network, one can obtain the exact same result that can be obtained by inserting the Server CD and installing apache2 from the CD-ROM.
• The Ubuntu Server Edition installation process is slightly different from the Desktop Edition. Since by default Ubuntu Server doesn't have a GUI, the process is menu driven, very similar to the Alternate CD installation process.
• Before 12.04, Ubuntu server installs a server-optimized kernel by default. Since 12.04, there is no difference in kernel between Ubuntu Desktop and Ubuntu Server since linux-image-server is merged into linux-image-generic.
• For Ubuntu LTS releases before 12.04, the Ubuntu Desktop Edition only receives 3 years of support. This was increased to 5 years in Ubuntu LTS 12.04 In contrast, all Ubuntu LTS Server Edition releases are supported for 5 years.

It's worth noting that other than the kernel settings, Ubuntu Desktop and Server are essentially the same distribution, just with different default package selection. They both use the same packages and respositories. If you run apt-get install ubuntu-desktop you will end up with the functional equivalent of Desktop Edition.

That also means that any package that's intended for Ubuntu Server will run just as happily on your desktop installation.

Whether you install using a server CD, or a desktop CD, you end up with the same Ubuntu. The difference is in what selection of packages it installs by default - that is - what software selection you end up with at the end of the installation process.

It is possible to move from a desktop system to a server system and vice-versa on an already-installed copy of Ubuntu. Ubuntu even makes it relatively easy with the tasksel utility or with meta-packages like ubuntu-desktop and ubuntu-server (available through the standard apt package manager at least as of 16.04). You can even mix and match - installing a desktop environment on a server or server software such as ssh_server or apache2 on a primarily desktop computer.

But chances are, you probably already know at install time whether you want a desktop system complete with desktop environment, or a server system. So having different installation CDs for server and desktop is simply a convenience factor that makes software selection just a bit simpler.

The installers also behave differently, in the sense that only the "desktop" version installs from a graphical Live CD. The other versions install using a menu-based installer similar to Debian's installer.