Projecting EPSG:4326 data in 2D map?

OpenLayers uses the term 'EPSG:4326' to mean the Plate Caree projection. Referring to 'WGS84' and EPSG:4326 as a projection has been common for so long that it is a source of confusion. This short-hand has been going on since before Google and OpenLayers came on to the scene. For instance, ESRI have been fudging the terms for as long as I can remember. OpenLayers does not reproject to EPSG:900913 on the fly unless you tell it to, which you have to if you want to mix data with Google base maps because the Google API only uses 900913 (which was invented by Google - the numbers being vaguely reminiscent of the word 'google').

EPSG:4326 is not a projection; it’s “unprojected”. Reykjavik is indeed at an angle relative to cardinal directions: http://osm.org/go/e0UtZkUl-, and when you look at your linked map you are seeing the lines of latitude and longitude mapped to a 2:1 rectangle that's 360° wide and 180° tall. The top and bottom edges are both points on the surface of the earth, the north and south poles.

On the web, a common projection used for lat/lon data is EPSG:900913 (or officially 3857), the Google spherical web mercator projection. That’s mostly just from the past few years when this projection became ubiquitous through web maps.