Why do we need to fork to create new processes?

The short answer is, fork is in Unix because it was easy to fit into the existing system at the time, and because a predecessor system at Berkeley had used the concept of forks.

From The Evolution of the Unix Time-sharing System (relevant text has been highlighted):

Process control in its modern form was designed and implemented within a couple of days. It is astonishing how easily it fitted into the existing system; at the same time it is easy to see how some of the slightly unusual features of the design are present precisely because they represented small, easily-coded changes to what existed. A good example is the separation of the fork and exec functions. The most common model for the creation of new processes involves specifying a program for the process to execute; in Unix, a forked process continues to run the same program as its parent until it performs an explicit exec. The separation of the functions is certainly not unique to Unix, and in fact it was present in the Berkeley time-sharing system, which was well-known to Thompson. Still, it seems reasonable to suppose that it exists in Unix mainly because of the ease with which fork could be implemented without changing much else. The system already handled multiple (i.e. two) processes; there was a process table, and the processes were swapped between main memory and the disk. The initial implementation of fork required only

1) Expansion of the process table

2) Addition of a fork call that copied the current process to the disk swap area, using the already existing swap IO primitives, and made some adjustments to the process table.

In fact, the PDP-7's fork call required precisely 27 lines of assembly code. Of course, other changes in the operating system and user programs were required, and some of them were rather interesting and unexpected. But a combined fork-exec would have been considerably more complicated, if only because exec as such did not exist; its function was already performed, using explicit IO, by the shell.

Since that paper, Unix has evolved. fork followed by exec is no longer the only way to run a program.

• vfork was created to be a more efficient fork for the case where the new process intends to do an exec right after the fork. After doing a vfork, the parent and child processes share the same data space, and the parent process is suspended until the child process either execs a program or exits.

• posix_spawn creates a new process and executes a file in a single system call. It takes a bunch of parameters that let you selectively share the caller's open files and copy its signal disposition and other attributes to the new process.

[I'll repeat part of my answer from here.]

Why not just have a command that creates a new process from scratch? Isn't it absurd and inefficient to copy one that is only going to be replaced right away?

In fact, that would probably not be as efficient for a few reasons:

1. The "copy" produced by fork() is a bit of an abstraction, since the kernel uses a copy-on-write system; all that really has to be created is a virtual memory map. If the copy then immediately calls exec(), most of the data that would have been copied if it had been modified by the process's activity never actually has to be copied/created because the process doesn't do anything requiring its use.

2. Various significant aspects of the child process (e.g., its environment) do not have to be individually duplicated or set based on a complex analysis of the context, etc. They're just assumed to be the same as that of the calling process, and this is the fairly intuitive system we are familiar with.

To explain #1 a little further, memory which is "copied" but never subsequently accessed is never really copied, at least in most cases. An exception in this context might be if you forked a process, then had the parent process exit before the child replaced itself with exec(). I say might because much of the parent could be cached if there is sufficient free memory, and I am not sure to what extent this would be exploited (which would depend on the OS implementation).

Of course, that doesn't on the surface make using a copy more efficient than using a blank slate -- except "the blank slate" is not literally nothing, and must involve allocation. The system could have a generic blank/new process template that it copies the same way,¹ but that would then not really save anything vs. the copy-on-write fork. So #1 just demonstrates that using a "new" empty process would not be more efficient.

Point #2 does explain why using the fork is likely more efficient. A child's environment is inherited from its parent, even if it is a completely different executable. For example, if the parent process is a shell, and the child a web browser, $HOME is still the same for both of them, but since either could subsequently change it, these must be two separate copies. The one in the child is produced by the original fork().

^{1. A strategy that may not make much literal sense, but my point is that creating a process involves more than copying it's image into memory from disk.}

I think the reason Unix had only the fork function to create new processes is a result of the Unix philosophy

They build one function that does one thing well. It creates a child process.

What one does with the new process is then up to the programmer. He can use one of the exec* functions and start a different program, or he could not use exec and use the two instances of the same program, which can be useful.

So you get a bigger degree of freedom since you can use

1. fork without exec*
2. fork with exec* or
3. just exec* without fork

and in addition you only have to memorize the fork and the exec* function calls, which in the 1970s you had to do.