Why is Bash everywhere (in most if not all Linux distributions)?

History (acquired not through research but through spending entirely too much time hanging out with Bell Labs people):

1. In the beginning (if you consider the beginning to be Unix Version 7) was the Bourne shell. Steve Bourne was the first to show that the shell which controlled the user's interaction could be a user program and not a special part of the operating system. A historic breakthrough. The shell itself was relatively clean for scripting, but had no command-line editing or job control. Bourne's Introduction to the Unix Shell is still useful for beginning users today.

   Edit: I've ignored some "prehistory" from Ken Thompson and John Mashey, also from Multics. I'm sure Bourne was aware of all this work (he was in the same lab, 1127, at Bell Labs), but Bourne's shell was definitive, and the earlier work had little influence except as interpreted by Steve Bourne. For example, although Ken later wrote the Plan 9 C compiler and was very influential on Plan 9, but Tom Duff's paper on the Plan 9 shell (rc) mentions only Bourne's shell, not Thompson's.

2. The shell is just a user program, so anybody can write one. As Version 7 Unix was being created in New Jersey, Berkeley Unix was being created in California. Bill Joy at Berkeley wrote csh, the C shell. Joy added job control and history, and later command-line editing, but was not aware of Bourne's work and so based his language on the Thompson shell (which I considered "prehistoric" in the previous bullet). The Unix community loved job control, but they also loved Bourne's language. For a not-particularly-good polemic against the csh language, see Csh Programming Considered Harmful. For a time, many people used csh interactively for its job-control and history features, but used Bourne's sh to write scripts. This situation was less than ideal.

   Edit: Thanks to DigitalRoss for straightening me out on the chronology of csh. Since I got my education from people who refer to BSD as "the Berkeley heresy", I was pretty short of facts there.

3. Dave Korn at Bell Labs did a brilliant re-engineering of the Bourne shell to produce the Korn shell (ksh). It was fully backward compatible with Bourne shell sh but provided a boatload of invaluable improvements. ksh became the basis of a POSIX standard and was shipped standard with Sun software. (This despite the fact that Bill Joy left Berkeley to help found Sun and was one of their leading software guys.)

4. Bell Labs and AT&T stupidly fail to make ksh open source. ksh88 is widely used, but having sources is not legal. Certain people become so addicted that they become digital criminals.

   Edit: Was this really so stupid? Hard to know. Berkeley was already giving Unix away, and other corporations were soon to follow, but this was still the era when the Corporate Masters believed in charging for Unix. But the results: AT&T Unix is dead, after having been sold off to various parties any number of times. BSD and its derivatives are alive and well, but these upstart things called "Linux" and "GNU" have a huge fraction of mindshare that once belonged to Bell Labs.

5. The Free Software Foundation does a "clean-room", from scratch implementation of a POSIX shell, taking all of Dave Korn's ideas as then current, plus in usual FSF style adding new features of their own, such as programmable completion. They call it the "Bourne again" shell, or bash.

6. In the middle 1990s AT&T open-sources ksh93, but by then it is too late for widespread adoption. The licensing agreement is weirdly nonstandard. bash and ksh diverge, and ksh never achieves market share commensurate with its place in history.

Lessons:

• The first adequate product to market wins (sh).

• People love new features (job control, command completion), but they love them even more when their old scripts continue to work.

• Edit: Professors of engineering should leave history to historians of science :-)

Bash has two completely different things going for it.

1. It's a fine shell. It is one of maybe 2 shells (the other is zsh) that integrate some of the cool csh features like ! history substitution into the posix syntax. It has lots of extensions, including arrays.

2. It is the FSF/GNU shell. In the open source world, this gives it a sort of cachet.

I should also add that it is not always the default. ash is often used as /bin/sh so that while bash may be the interactive shell, ash is the "just run the command file" shell. This is because ash is smaller and faster, and contains the posix features, so it's a proper subset. Using ash as an interactive shell is sometimes problematic. On, say, NetBSD, it works well, because there it is built with all the features. It's kind of their one shell whereas bash is an external package. But on Linux ash is usually considered non-interactive, so they compile it without the history and without the (important) line editing on the theory that it is just used to run those huge gnu configure scripts.

Tale of Two Shells

The real history of the shell

UPDATE: There is an inaccurate history of the shell being copied from place-to-place on the web, and people are understandably believing it. I will try to give an accurate version and provide a few links to substantiate it here.

1. The first shell was most certainly not the Bourne shell but was written by Ken Thompson himself and distributed in V6, which is the version AT&T sent to various universities and government labs. This is what put Unix on the map. It had all the basics, <, >, >>, |, &, but it just had simple goto control syntax via an external program that seeked on standard input. There were no complex shell scripts then. Later shells would open the command input on a separate fd. It may look simple today but in the horror-movie that was 1970's computing it was the best thing on earth. Believe it or not, this ancient shell has its own twitter stream today and, of course, a home page.
2. The second shell was csh, written (as was vi) by Bill Joy at UCB. This was before GNU readline and NetBSD editline, so it must have seemed perfectly reasonable to do history with the ! syntax. Csh added most of today's shell features but with csh syntax. csh did not change any syntax, gratuitously or otherwise. It was actually backwards compatible to the Thompson shell, and originally included TS source code.
3. The third shell was the Bourne shell, with different syntax. Unix was being developed in parallel at UCB and AT&T. This shell had a weird memory allocator (I think it just used more memory, trapped SIGSEGV, did a new brk(2), and then tried again) that made it hard to run on new Unix ports, so osh and csh stayed popular for some time. There was no internet and it was licensed SW, so in that environment it's possible that Stephen Bourne didn't know about Joy's shell and certainly Joy didn't know about Bourne. It's possible that the two shells first met when UCB got a VAX and a prerelease of the now-forgotten Unix/32V. I remember Bill complaining about the memory allocation. Note that both shells were backward-compatible to the V6 shell, they simply extended the syntax in different directions.
4. Now there really were multiple incompatible shells, to which AT&T added the Bourne-compatible ksh. Eventually, csh had semi-available source code, but it was tied up in a lawsuit between AT&T and the University of California. Still, these were the glory days of BSD Unix as sophisticated companies who could afford the $50,000 fee would buy the AT&T license but install the 4.x BSD distributions, and universities got it for free.
5. In this situation with many legal and technical issues, various independent implementations were undertaken. At least as many went with the csh syntax as went with the Bourne shell syntax, and some merged the two. You had at least tcsh, zsh, bash, and ash. The Bourne syntax was "official", being part of AT&T releases, but in those days BSD was quite important, and Sun, initially BSD, distributed a fair amount of the Unix SW that the world encountered.
6. Partly because of the USL lawsuit, the FSF and Linux had an open field. Meanwhile, AT&T had managed to pick a fight with one of the few entities on earth larger than they were (The State of California) and in the end they didn't win the suit, and so eventually the BSD distribution was on a firm legal footing. But by then Linux and bash were everywhere, and so today BSD is a niche.
7. Finally, bash is a good shell (though apparently disowned privately by its original author) and it fully deserves the credit for its own success. csh would have been eclipsed by tcsh and zsh even if ash, bash, and ksh had not won the syntax war.

To add to what @DigitalRoss said

• Bash is a complete superset replacement for posix-sh, even to the point if called as /bin/sh will emulate posix-sh entirely. Posix-sh was the "standard" for commercial unix systems as a common-denominator shell. So, something that starts there and builds on it is starting with a lot.