Where does Mac OS X come from?

The history of MacOS is a little bit more convoluted. I was very interested in this in the late 90's as Mach had been pitched around the world as a faster way of building a Unix system.

The origin of the kernel is a bit more complicated.

It all starts with AT&T distributing their operating system to some universities for free. This Unix was improved extensively at Berkeley and became the foundation for the BSD variations of Unix and incorporated several new innovations like the "Fast File System" (UFS), introduced symlinks and the sockets API. AT&T went on their own way and built System V at the same time.

Meanwhile, research continued and some folks adopted the work from BSD as a foundation. At CMU, the BSD kernel was used as the foundation for prototyping a few new ideas: threads, an API to control the virtual memory system (through pluggable "pagers" - user level mmap), a kernel-level remote procedure call system and most importantly the idea of moving some kernel level operations to user space. This became the Mach kernel.

I am not 100% sure if mmap came from Mach, and later was adopted by BSD, or if Mach merely pioneered the idea and BSD added their own mmap based on the ideas of Mach.

Although the Mach kernel was described as a micro-kernel, up to version 2.5 it was merely a system that provided the thread, mmap, message passing features but remained a monolithic kernel, all the services were running on kernel mode.

At this time Rick Rashid (now at Microsoft) and Avie Tevanian (now at Apple) had come up with a novel idea that could accelerate Unix. The idea was to use the mmap system call to pass data to be copied from user space to the "servers" implementing the file system. This idea was essentially a variation of trying to avoid making copies of the same data, but it was pitched as a benefit of micro kernels, even if the feature could be isolated from a micro kernel.

The benchmarks of this VM-backed faster Unix system is what drove people at NeXT and at the FSF to pick Mach as the foundation for their kernels.

NeXT went with the Mach 2.5 kernel (which was based on either BSD 4.2 or 4.3) and GNU would not actually start on the work for years. This is what the NeXTSTEP operating systems were using.

Meanwhile at CMU, work continued on Mach and they finally realized the vision of having multiple servers running on top of a micro kernel with version 3.0. I am not aware of anyone in the wild being able to run Mach 3.0 as all of the interesting user-level servers used AT&T code so they were considered encumbered, so it remained a research product.

Around this time the Jolitz team had done a port of 4.3+ BSD to the 386 architecture and published their porting efforts on DrDobbs. 386BSD was not actively maintained and a group emerged to maintain and move 386BSD forward, the NetBSD team. Internal fights within the NetBSD group caused the first split and FreeBSD was formed out of this. NetBSD at the time wanted to focus on having a cross-platform BSD, and FreeBSD wanted to focus on having a Unix that did great on x86 platforms. A little bit later, NetBSD split again due to some other disputes and this lead to the creation of OpenBSD.

A fork of BSD 4.3 for x86 platforms went commercial with a company called BSDi, and various members of the original Berkeley team worked there and kept good relations with the BSD team at the University.

AT&T was not amused and started the AT&T vs BSDi lawsuit, which was later expanded to sue the University as well. The lawsuit was about BSDi using proprietary code from AT&T that had not been rewritten by Berkeley. This set back BSD compared to the up and coming Linux operating system.

Although things were not looking good for the defendants, at some point someone realized that SystemV had incorporated large chunks of BSD code under the BSD license and AT&T had not fulfilled their obligations in the license. A settlement was reached in which AT&T would not have to pull their product from the market, and the University agreed to rip out any code that could still be based on AT&T code.

The university then released two versions of BSD 4.4 encumbered and 4.4 lite. The encumbered version would boot and run, but contained AT&T code. The lite version did not contain any code from AT&T but did not work.

The various BSD efforts re-did their work on top of the new 4.4 lite release and had a booting system within months.

Meanwhile, the Mach 3.0 micro kernel remained not very useful without any of the user-land servers.

A student from a Scandinavian university (I believe, I might have this wrong) was the first one to create a full Mach 3.0 system with a complete OS based on the 4.4 lite release, I believe this was called "Lites". The system worked, but was slow.

During the 1992-1996 and by now BSD already had an mmap() system call as well as most other Unix systems. The "micro kernel advantage" that was not there, never really came to fruition. NeXT still had a monolithic kernel. The FSF was still trying to get Mach to build, and not wanting to touch the BSD code or contribute to any of the open source BSD efforts, they kept charging away at a poorly specified kernel vision and they were drowning on RPC protocols for their own kernel. The micro kernel looked great on paper, but turned out to be over engineered and just made everything slower.

At this point we also had the Linus vs Andy debate over micro-kernels vs monolithic kernels and the world started to realize that it was just impossible to add all of those extra cycles to a micro kernel and still come ahead of a well designed monolithic kernel.

Apple had not yet acquired NeXTSTEP, but was also starting to look into Mach as a potential kernel for their future operating systems. They hired the Open Software Foundation to port Linux to the Mach kernel, and this was done out of their Grenoble offices, I believe this was called "mklinux".

When Apple bought NeXT what they had on their hands was a relatively old Unix foundation, a 4.2 or 4.3 based Unix and by now, not even free software ran well out of the box on those systems. They hired Jordan Hubbard away from FreeBSD to upgrade their Unix stack. His team was responsible for upgrading the user land, and it is not a surprise that the MacOS userland was upgraded to the latest versions available on BSD.

Apple did switch their Mach from 2.5 to 3.0 at some point, but decided to not go with the micro-kernel approach and instead kept everything in-process. I have never been able to confirm if Apple used Lites, hired the scandinavian hacker, or if they adopted the 4.4 lite as their OS. I suspect they did, but I had already moved on to Linux and had stopped tracking the BSD/Mach world.

There was a rumor in the late 90's that Avie at Apple tried to hire Linus (who was already famous at this point) to work on his baby, but Linus chose to continue working on Linux.

History aside, this page describes the userland and the Mach/Unix kernel:

http://developer.apple.com/mac/library/documentation/Darwin/Conceptual/KernelProgramming/Architecture/Architecture.html#//apple_ref/doc/uid/TP30000905-CH1g-CACDAEDC

I found this graphic of the history of OSX:

On the Unix side, OS X is a descendant of NeXTSTEP, which was derived from 4.3BSD with the core parts of the kernel replaced with Mach.

The NeXT programming API, which eventually came to be called OpenStep, is the basis of today's Cocoa API for OS X. Two APIs have diverged greatly since Apple bought NeXT in 1997, though there are ongoing efforts to provide open source API-compatible Cocoa clones.

Add to that the Classic MacOS compatibility API, called Carbon, and you have the OS X programming interface.

(There's plenty more to OS X, but they're applications on top of all this: Finder, the BSD and GNU userland tools, etc..)

As for the FreeBSD kernel idea, it's sorta correct, but it's an unsophisticated way to look at it. The original kernel came, as I said, from NeXT, which assembled their first kernel from 4.3BSD and Mach. This means that both FreeBSD and NeXTSTEP shared some code via 4.3BSD.

The meme that OS X is based on FreeBSD has two more recent sources. First, Apple has continued to borrow innovations from the BSD world, usually from FreeBSD. Second, Apple hired FreeBSD project co-founder Jordan Hubbard shortly after making the first public OS X release. He worked for Apple through June 2013.