This interpretation of permissions dates back to early Unix file systems. In the beginning, there were only files. (Well, and devices, and pipes, and... but I'm trying to tell a story here, not be 100% strictly accurate; besides, its all true for devices and pipes and everything else because everything is a file, even directories).
Directories are just files that the file system uses to hold the metadata that describes the directory tree, and the files it contains. Each file in a directory was described by a simple data structure that contained room for a file name (originally 14 characters, IIRC) along with the inode number where the data was stored, the size of the file, time stamps, and the permissions word. Every directory began with two entries named . and .., the first pointing at the inode of this very directory, and the second at the inode of its parent directory.
The permissions word had nine bits to describe the treatment of the owner, other members of the same group, and the world. The three bits for each flag whether the relevant user can read, write, or execute the file. (You might notice that there are five more bits in the 16-bit permissions word that I am ignoring. Those did eventually get meanings assigned but that isn't relevant to this part of the story.) (Also, this interpretation of the nine bits has remained pretty much the same in all descendents of early Unix, including Linux.)
So, if a directory is really just a special kind of file and described by an entry in some directory, it obviously also has permission bits, and those bits probably mean something. But the question is what, exactly. The easiest way to assign meaning to those bits is to not change what they mean in the first place. And that is essentially what was done.
So, the read bit means that the user can read the directory itself. That classically gives the reader access to the file name, time stamps, size, and inode number of each file's data. Specifically, with r set you can use ls to see the names of all the files in the directory, but that isn't sufficient to open (or use in any way) any of the listed files.
The execute bit means that the user can "execute" the directory. Since directories are special, execute really means look up an entry by name and use it. That means that you can try to open the files if x is set, but without r you can't discover their names. Of course, the requested file's permissions also affect access, so even with x on the directory you won't be able to read a file unless it also offers you r.
The write bit means that the user can write to the directory, but naturally only mediated by the file system itself. That means that with w set you can create new files in that directory, or edit the directory entries of existing files. But without x set, you can't actually use any files, and without r you can't see them either.
As more complicated models of user identity have evolved in Unix and its descendents, these same basic descriptions have managed to stay remarkably unchanged.
In short, r means you can see its content, x means you can use it, and w means you can modify it even for directories.
