What's so great about ARM?
Performance is one advantage. Being a 32-bit processor it outperforms (almost) all 8-bit controllers DMIPS-wise. The core also has gone through several generations, read optimizations.
These optimizations not only show in performance numbers, but in power consumption as well. The most recent core has doubled its DMIPS/mW ratio compared to the previous generation (see also this answer).
ARM is available from a great many manufacturers, more than any other microcontroller, and each has a number of versions to choose from, with different combinations of on-chip peripherals and memory, and packages. Case in point: NXP offers no less than 35 controllers with on-chip Ethernet.
ARMs are inexpensive; ARM was probably the first 32-bit controller to break the USD 1 barrier.
This combination of performance, wide offering and low cost make it such that you simply can't ignore ARM:
In 2005 about 98 percent of all mobile phones use at least one ARM-designed core on their motherboards, according to research from the analyst firm the Linley Group. (source)
The mobile phone market has also another effect. Mobile phones are very space constrained and demand small packages. NXP's LPC1102 comes in a WLP-16 package just 5mm\$^2\$, a scale previously only used by low pin-count 8-bit microcontrollers.
One point not yet mentioned: In 1908, a guy named Mr. P.L. Robertson invented a new improved screw head and driver. He wanted to be the only person to manufacture screws and drivers to his design. Decades later, someone else named Mr. Henry F. Phillips came up with an alternative design. Unlike Mr. Robertson, Mr. Phillips was more interested in licensing his design than in producing screws and drivers.
Likewise, in the 1970s, Sony came up with a technology called Betamax; JVC came up with one called VHS. Sony was interested in producing VCRs; JVC was more interested in licensing.
There seems to be a pattern here. (Note: For awhile, Intel did license its 80x86 technology, but it has for decades been focused more on developing technologies for its internal use.)
Same hardware/software to develop for ARM's of all vendors. You buy JLINK/ULINK and some Keil IDE once, and can use it to develop, emulate and debug nearly any ARM on a planet.
No need to learn new architecture when moving to new chip vendor => less vendor lockin => more competition => lower prices
In modern tech-processes(0.18um and below) ARM core is so tiny, that sacrificing it for 8bit core would not save any visible fraction of a price. Hence the reason of going for standard high-performance yet cheap architecture.
Performance - only with ARM you can have single clock 32*32->32 multiplication and hardware support for 32*32->64 and division for sub-1dollar devices (namely, lower-end STM32's for example)
ARM is not too greedy, and does not charge unreasonable fees for licenses, so manufacturers can produce cheap micros.