How are users able to execute a file without permission?

In your examples, you are not executing the files, but sourcing them.

Executing would be via

$ ./hello.sh

and for that, execution permission is necessary. In this case a sub-shell is opened in which the commands of the script file are executed.

Sourcing, i.e.

$ . hello.sh

(with a space in between) only reads the file, and the shell from which you have called the . hello.sh command then executes the commands directly as read, i.e. without opening a sub-shell. As the file is only read, the read permission is sufficient for the operation. (Also note that stating the script filename like that invokes a PATH search, so if there is another hello.sh in your PATH that will be sourced! Use explicit paths, as in . ./hello.sh to ensure you source "the right one".)

If you want to prevent that from happening, you have to remove the read permission, too, for any user who is not supposed to be using the script. This is reasonable anyway if you are really concerned about unauthorized use of the script, since

• non-authorizeded users could easily bypass the missing execution permission by simply copy-and-pasting the script content into a new file to which they could give themselves execute permissions, and
• as noted by Kusalananda, otherwise an unauthorized user could still comfortably use the script by calling it via

  sh ./hello.sh
  

  instead of

  ./hello.sh
  

  because this also only requires read permissions on the script file (see this answer e.g.).

As a general note, keep in mind that there are subtle differences between sourcing and executing a script (see this question e.g.).

The execute permission simply means that this file can be executed. However, when you source it (. hello.sh or source hello.sh) or when you pass it as an argument to a shell interpreter (sh hello.sh), you aren't executing the file, you're executing another command (. or sh) and passing the file as an argument to that command.

So to answer your question, the reason you can "execute" the file with . hello.sh is the same reason why you can run cat hello.sh: you are just reading the file, not executing it.

To illustrate:

$ ls -l
total 4.0K
-r--r--r-- 1 terdon terdon 21 May 29 12:29 foo.sh

$ cat ./foo.sh 
#!/bin/sh
echo Hello

$ ./foo.sh
bash: ./foo.sh: Permission denied

$ sh ./foo.sh
Hello

As you can see, I cannot actually execute the script, but I can read it—either with cat or with .—perfectly well.

The very short version is: you are not executing the file. You are reading it into the shell, which then executes it.

Note that some language interpreters check the execute permissions of a file they are being asked to execute and will refuse to do so if the user doesn't have appropriate permissions. But that is a completely discretionary check by the author of that particular interpreter, and is not enforced by the operating system.

Let's make a little thought experiment.

I think we can both agree, that if a program has executable permissions, I should be allowed to execute this program. I think we can also agree that this program should be allowed to print the string "Hello World" to the console.

Further, we can probably also agree that a program should be able to read a file as long as that file has read permissions. And it doesn't really matter what the content of that file is, as long as the file has the read permission bit set, the program should be allowed to read it.

Okay, so, since we have agreed that the program should be allowed to print "Hello World" to the console and we have agreed that the program should be allowed to read the file, you must logically also agree that the program should be allowed to read the file and print "Hello World" to the console. And we have also said that whether or not the program is allowed to read the file depends on the permissions, and not on the content, so it should also be allowed to read the file and print "Hello World" to the console if the content of the file is echo Hello World.

Now, can you also agree that since the program is allowed to read the file and since the program is allowed to print "Hello World" to the console, that it is also allowed to check the content of the file and only print "Hello World" to the console if the content of the file is echo Hello World?

Well, but then it is "interpreting", in other words executing the file, and you have just agreed with me on every step of the way that it should be allowed to do that!

And that is exactly what is happening here. The shell is simply reading a text file, which it is allowed to do because the text file has read permissions. And the shell is executing the instructions in the text file, which it is allowed to do because the shell has execute permissions.