What does the '!' really do when it's added to an ex command (:wq! | :w! | :q! )?

! generally means what you'd expect from "force", but what it means for specific command depends on the command. In the case of w!, if Vim cannot write to the file for some reason, it will try to delete and create a new one with the current buffer's contents.

Consider the following example (observe the inode numbers):

$ touch foo
$ chmod -w foo
$ stat foo
  File: ‘foo’
  Size: 0           Blocks: 0          IO Block: 4096   regular empty file
Device: 22h/34d Inode: 10396141    Links: 1
Access: (0444/-r--r--r--)  Uid: ( 1000/ muru)   Gid: ( 1000/ muru)
Access: 2015-09-10 00:24:28.259290486 +0530
Modify: 2015-09-10 00:24:28.259290486 +0530
Change: 2015-09-10 00:24:30.771263735 +0530
 Birth: -
$ vim -c 'r!date' -c 'wq!' foo
$ stat foo                    
  File: ‘foo’
  Size: 30          Blocks: 8          IO Block: 4096   regular file
Device: 22h/34d Inode: 10396151    Links: 1
Access: (0444/-r--r--r--)  Uid: ( 1000/ muru)   Gid: ( 1000/ muru)
Access: 2015-09-10 00:24:37.727189657 +0530
Modify: 2015-09-10 00:24:37.731189614 +0530
Change: 2015-09-10 00:24:37.763189273 +0530
 Birth: -
$ cat foo
Thu Sep 10 00:24:37 IST 2015

That's why the owner and group changes. Permissions are preserved - :h write-permissions:

                                                    write-permissions
When writing a new file the permissions are read-write.  For unix the mask is
0666 with additionally umask applied.  When writing a file that was read Vim
will preserve the permissions, but clear the s-bit.

If you want to make Vim refuse writes, see :h write-readonly:

                                                    write-readonly
When the 'cpoptions' option contains 'W', Vim will refuse to overwrite a
readonly file.  When 'W' is not present, ":w!" will overwrite a readonly file,
if the system allows it (the directory must be writable).

Note that it says "the directory must be writable" - because without a writable directory, Vim can neither delete nor create a new file.

As stated in vim(1), -R ensures that the file will not be accidentally overwritten when the user blindly says :w, but doesn’t disable writing with :w!.

But it’s only an application, and when about to actually do something with files, the OS kernel will check permissions anyway. An experiment: I ran

strace vim /etc/hostname 2>vim.out

under a user. Even without explicit -R it started in readonly, because it looked at permissions. After changing the buffer

W10: Warning: Changing a readonly file

appeared.

Now :w and I got

E45: 'readonly' option is set (add ! to override)

I applied the suggestion and

"/etc/hostname" E212: Can't open file for writing

Predictably, in vim.out we see:

open("/etc/hostname", O_WRONLY|O_CREAT|O_TRUNC, 0644) = -1 EACCES (Permission denied)