What is X Window System?
You may be confused, and this is not your fault, because Linux can have 2 meanings.
Linux is a kernel: This kernel is used in many systems, including android and the systems outlined in 2.
Linux also often confusingly is used to refer to systems like Debian, Ubuntu, Redhat, CentOs, Suse, and many more. These systems are better described as Gnu+Linux, and in the desktop case X11+Gnu+Linux.
X11 is the correct name for the system that you are asking about. X11 is an architecture independent, network transparent, policy free, windowing system.
Not part of the OS
The X11 server runs as a user process. Other processes also run, window manager (to decorate windows with frames and title bars, and do the moving and resizing), taskbar, and other. It has been said “MS-Windows is a windowing system with an OS stuck on as an after thought, and Unix/Linux is an OS with a windowing system stuck on as an after thought.”
Architecture independent
X11 is used on most Unixes: Gnu (including Gnu+Linux), Bsd, Solaris, Hp-Ux, etc. it can also be used on many other systems: Vms, MS-Windows, MacOs, AmigaDos, and may more.
Network transparent
You can open windows on other machines, if you have the correct authority. And can run application on a remote (possibly more powerful) machine, and display application locally. This is done on a per application, or per window basis, unlike VNC or remote desktop, that do it a desktop at a time.
Policy free
X11 has no policy as to what things look like or how things are done. This has allowed it to endure since about 1985, with some extensions such as video, shaped windows and 3d. All the changes of look and feel are done by changing or replacing window managers and other helper apps. You can change window manager without logging out. So it is possible to change from something from 1985 (pre Microsoft's windows) to 1995 (Win95), to 2001 (win XP), to 2014 (Win 7) to (no one in there right mind would run something like win 8), to something better — no reboot, not even a log out. (Though if you did have a 1985 version of X11 you would have to logout to upgrade to one that has 3D etc).
see also http://en.wikipedia.org/wiki/X_Window_System
The X Window System is the basis for graphical user interfaces. X uses a client-server model: the X server is responsible for displaying (drawing) everything, while the client tells the X server what to display (draw). Client and server do not need to be on the same host.
X by itself is not sufficient for graphical user interfaces, for the X server "knows" only how do draw rectanges, lines, bitmaps and other shapes and how to render fonts. It also takes care of window overlapping, cursors, input event management (such ad key press and release, mouse movements, mouse clicks) etc. It does not provide any kind of "look and feel". That's up to widget sets that operate on top of X. Examples for such sets are Qt or GTK. Historical examples are Motif and XView. These widget sets "describe" what widgets exist (like text input fields, drop down boxes, buttons, canvases etc), how they look like, and how they act. On top of that there are the window managers which are responsible for, well, window management, i.e. how windows are decorated (frames, title bars) and what action elements they provide (like minimize, maximize, close etc.)
Unlike Microsoft Windows, X is not part of the operating system. Although the X server used to have extraordinary privileges in order to utilize the graphics hardware, you can run a unix/linux system perfectly without X. However, a lot of software depends on the X libraries, so even if the X window system is not fully installed on a system, you might find the X libraries.