Is there some kind of icon browser?

Instead of opening the folders one by one, you can also use the Search feature of Nautilus. Navigate to /usr/share/icons, and press the Search icon on the right of the toolbar.

Search for . (all images have an extension with a dot before it) and press Enter. On a default installation, this yields about 17.5k images. That's not much of an "overview", but it includes all files in /usr/share/icons.

If you wish to avoid searching each time, you could make use of symbolic links to the images: all images are accessible from one big folder.

1. Open a terminal

2. To check the number of files that can be created after creating the links:

   expr $(df /home -i | tail -1 | cut -d'%' -f1 | rev | awk '{ print $2 }' | rev) - $(find /usr/share/icons -type f | wc -l)
   

   You should not continue if the number is lower than 1000 and a negative number will cause the operation to fail after some time.

3. Make an folder named icons-all by running: mkdir icons-all
4. Go into that folder: cd icons-all
5. Run nano /tmp/make-icons-link

6. Paste:

   #!/bin/bash
   if [[ $1 == *.* ]]; then
           ext=".${1##*.}"
   else
           ext=
   fi
   name="$(basename "$1" "$ext")"
   extra=
   while [ -e "$name$extra$ext" ]; do
       ((extra++))
   done
   ln -s "$1" "$name$extra$ext"
   

7. Press Ctrl + X, followed by Y and Enter

8. Now generate the links, this may take a while:

   find /usr/share/icons/ -type f -exec bash /tmp/make-icons-link {} \;
   

   After the command has completed, no output is shown.

9. Close the terminal by running exit
10. The images are now visible in ~/icons-all. Loading this directory may take a while