Why does the original Game Boy support four shades of grey?
The first computers were monochrome; they only knew 2 colors, as for each pixel, you only had 1 bit:
0
1
As for the Gameboy, it supported 2 bits per pixel, which gives us 4 possible colors to choose from:
00
01
10
11
How did the makers decide on four and why not three
Because they already had 4 colors to choose from. When people have 10 fingers, why not count up to 10?
or five?
Because that would've required they increase the screen's bit depth to 3 bits per pixel, which would have given us 8 colors to choose from. Why didn't they do it? Probably for performance reasons. As you increase the amount of bits per pixel, you increase the amount of data to process, which in turn requires more powerful hardware to process.
Was there a precedent to using four shades of grey in computing graphics before the Gameboy, or was this an invention of the system?
Of course there were precedents.
CGA, gray-scale early NeXTstation, color Macintoshes, Atari ST.
While Nolonar's answer is true from a technical perspective, it's not the actual reason why the developers chose to go with a monochrome system. Gunpei Yokoi, the creator of the Game Boy, intentionally designed the system monochrome to force games to be more abstract, thus preventing players from getting too caught up in details. As he put in in a 1997 interview:
The technology was there to do color. But I wanted us to do black and white anyway. If you draw two circles on a blackboard, and say “that’s a snowman”, everyone who sees it will sense the white color of the snow, and everyone will intuitively recognize it’s a snowman. That’s because we live in a world of information, and when you see that drawing of the snowman, the mind knows this color has to be white. I became confident of this after I tried playing some Famicom games on a black and white TV. Once you start playing the game, the colors aren’t important. You get drawn, mentally, into the world of the game.
The full interview can be read here: http://www.techspot.com/news/61318-console-gaming-now-fascinating-1997-interview-nintendo-legendary.html