Why can't I list a directory with read permissions?

Some preparations, just to make sure that ls does not try more things than it should:

$ unalias ls 2>/dev/null
$ unset -f ls
$ unset CLICOLOR

Demonstration of the r directory permission:

$ ls -ld d
dr--------  3 ccorn  ccorn  102  4 Okt 14:35 d
$ ls d
f
$ ls -l d
ls: f: Permission denied
$ ls -F d
ls: f: Permission denied

In traditional Unix filesystems, a directory was simply a list of (name, inode number) pairs. An inode number is an integer used as index into the filesystem's inode table where the rest of the file metadata is stored.

The r permission on a directory allows to list the names in it, but not to access the information stored in the inode table, that is, getting file type, file length, file permissions etc, or opening the file. For that you need the x permission on the directory.

This is why ls -l, ls -F, ls with color-coded output etc fail without x permission, whereas a mere ls succeeds.

The x permission alone allows inode access, that is, given an explicit name within that directory, x allows to look up its inode and access that directory entry's metadata:

$ chmod 100 d
$ ls -l d/f
-rw-r--r--  1 ccorn  ccorn  0  4 Okt 14:35 d/f
$ ls d
ls: d: Permission denied

Therefore, to open a file /a/b/c/f or list its metadata, the directories /, /a, /a/b, and /a/b/c must be granted x permission.

Unsurprisingly, creating directory entries needs both w and x permissions:

$ chmod 100 d
$ touch d/g
touch: d/g: Permission denied
$ chmod 200 d
$ touch d/g
touch: d/g: Permission denied
$ chmod 300 d
$ touch d/g
$

Wikipedia has a brief overview in an article on file system permissions.

To read a directory you also need to be able to traverse into it (the x bit). So, at minimum you need r-x for a directory to be able to access it in any way.