Why are POSIX mandatory utilities not built into the shell?
Why are POSIX mandatory utilities not built into shell?
Because to be POSIX compliant, a system is required1 to provide most utilities as standalone commands.
Having them builtin would imply they have to exist in two different locations, inside the shell and outside it. Of course, it would be possible to implement the external version by using a shell script wrapper to the builtin, but that would disadvantage non shell applications calling the utilities.
Note that BusyBox took the path you suggested by implementing many commands internally, and providing the standalone variant using links to itself. One issue is while the command set can be quite large, the implementations are often a subset of the standard so aren't compliant.
Note also that at least ksh93, bash and zsh go further by providing custom methods for the running shell to dynamically load builtins from shared libraries. Technically, nothing then prevents all POSIX utilities to be implemented and made available as builtins.
Finally, spawning new processes has become quite a fast operation with modern OSes. If you are really hit by a performance issue, there might be some improvements to make your scripts run faster.
1 POSIX.1-2008
However, all of the standard utilities, including the regular built-ins in the table, but not the special built-ins described in Special Built-In Utilities, shall be implemented in a manner so that they can be accessed via the exec family of functions as defined in the System Interfaces volume of POSIX.1-2008 and can be invoked directly by those standard utilities that require it (env, find, nice, nohup, time, xargs).
Shell scripts are not expected to run with that type of speed. If you want to improve the speed of your script, try it in perl. If that is still too slow, then you'll have to move to a statically typed language such as java or c, or write a C module for perl that runs the parts which are too slow.
Shell is the first level of prototyping, if you can prove the concept with shell, then move to a better scripting language which can do more bounds checking which would take acres of shell.
A Unix OS is expected to include many small programs which do well defined tasks which make up a larger picture. This is a good thing as it compartmentalises bigger programs. Take a look at qmail, for example and compare that with sendmail. qmail is made of many programs:
http://www.nrg4u.com/qmail/the-big-qmail-picture-103-p1.gif
Exploiting the network daemon would not help you exploit the queue manager.
From the BASH reference manual,
Builtin commands are necessary to implement functionality impossible or inconvenient to obtain with separate utilities.
As I'm sure you've heard, the UNIX philosophy relies heavily on multiple applications that all have limited functionality. Each built-in has a very good reason why it is built in. Everything else is not. I think a more interesting class of questions is along the lines of, "why exactly is pwd built-in?"