Is it possible to compile a Darwin Distro from source, like you can a build a Linux Distro?

Update 19 Apr 2015:

After two years there still appears to be very little interest in this sphere. However, the Hackintosh community is still highly active, meaning that one of the few open source non-Apple bootloaders capable of booting xnu (Chameleon and forks) is still maintained and can boot Yosemite. There are also success stories booting OS X Yosemite within QEMU. Furthermore, thanks to a (now Apple-employed) developer who goes by the handle winocm, we have an ARM port of the xnu kernel. She has been the most active developer that I know of in this area.

There is also a sequel to Amit Singh's Mac OS X Internals coming soon. I usually don't like to mention people's personal pages; however, the blog server with all the info appears to be a little unreliable, so see the info box on ameaijou's Twitter page.

I have managed to build Apple's development toolchain (a self-host, however the "Darwin SDK" has also been ported to Linux). I believe that a Darwin OS may yet be possible to build from scratch – about all we might be missing are some opens-source Kexts. Watch this space, and if you know how to drum up interest, let me know! :)

Short answers to this question:

Technically: Yes

Practically: No*

With Binary Cheats: Probably, but also not legal (untested)

With Binary Cheats for generic hardware: As above (untested)

*unless you work at Apple (*clears throat in general direction of California*)

Longer answer:

This is going to be rather long. I suggest coffee. If you don't have the time/inclination to read it all you can skip to "Concluding Remarks".

Practically Possible (No):

Sadly, Apple has withdrawn the source code for too many of Darwin's necessary KEXTs and binaries to make the compilation of a Darwin OS purely from source possible. It still is technically possible (you could write the source yourself to patch it up properly), but I simply don't have the time, skills or inclination to do this (and I doubt the crowd funding community would be very interested).

Unsurprisingly the key tipping point was the release of Darwin 10, which brought xnu into x86_64-land. Most of the necessary sources were around before then, but were x86 only. As time has gone on, the meaning of Apple's "Open Source" appears to have shifted to "Open Source on Apple's hardware only" as Apple's KEXTs are now by-and-large hardware-specific, so even if you could get everything up and running (see below) you'd still be confined to Apple hardware.

Technically Possible (Yes):

However, all is not lost. The LFS guide did come in handy and certainly all of the necessary setup can be conducted without actually building the Darwin OS. Furthermore the steps presented give you an almost-exact roadmap of the path ahead, minus the kernel, KEXTs and bootloader. I did manage to solve the bootloader problem, though (at least for Apple hardware).

If you're interested, here's a full overview of what you will have to do:

  • Erase a partition (8 GB or larger preferably) on a drive (internal or external – doesn't matter) and format it as Mac OS Extended (Journaled) (HFS+).

  • Ensure it has a GUID Partition Table (GPT) and that when you do it has an EFI partition. The easiest way to do this is through Apple's Disk Utility, but you can do it on the command line if you want (there are tutorials elsewhere on how to do this). The important point is that when you run distil list diskNsM, the following information should be correct:

    Partition Type: Apple_HFS

    OS Can Be Installed: Yes

    Read-Only Media: No

    Read-Only Volume: No

  • Now to follow the LFS guide (with adaptations).

  • Insert DFS=/Volumes/DarwinOS (using the actual mount point, obviously) into .bashrc and .bash_profile

  • Make the user directory (chown it to 0:0 at the very end):

    sudo mkdir -v "$DFS"/usr

  • Enter root:

    sudo su -

  • Make the sources directory and set the sticky bit:

    mkdir -v "$DFS"/sources     # Make sure you still have $DFS defined; if not, redefine it.
chmod -v a+wt "$DFS"/sources

  • Make the tools directory and make a symlink to it so we can add it easily to $PATH later (still in root by the way):

    mkdir -v "$DFS"/tools
ln -sv "$DFS"/tools /
logout          # Leave root

  • Download the source of all the packages you want. This is, of course, where you get stuck. All the necessary ones aren't there. (Incidentally, I prefer GNU's binutils anyway.)

Assuming you could in fact download all the ones you needed, let's press on.

Okay now to the hypothetical section.

From here out it's pretty well going to be standard LFS procedure: Extract sources, build, install, test, delete sources. Note the 2 passes of binutils, GCC and Glibc are still necessary BUT you ALSO have to have a working copy of libc++.1.dylib – and you'll need to do that in 2 passes as well. You can see the LLVM project's libcxx page for a few more details. Once compiled, you can put it in /usr/lib. You'll need to compile and install the xnu kernel (there are a few tutorials around the web for how to do that) and then install the KEXTs. Even if all the required KEXTs were available, you'd still need to put them into the .kext package by hand. Again there are tutorials for how to hand-craft a KEXT on the command line.

The final bit is making the system bootable. To do that, you would run the following command:

 "$DFS/usr/sbin/bless" --folder "$MOUNT/System/Library/CoreServices" --bootefi --verbose

Actually the location to bless doesn't really make a difference. That folder is just Apple-standard.

In any case, assuming the kernel and KEXTs were in the right places, you had proper copies of dyld, launchd, etc in place and boot.efi was working correctly, the system should be working and bootable!

Note if you really wanted to you could run a faux-launchd that is just a script to run a bash prompt – that's what PureDarwin Nano does.

Again, by all means, write the KEXTs and binaries yourself if you like – it is technically possible. Just call me when you're done.

With Binary Cheats: Probably, but also not legal (untested)

So, why can't you just rip the binaries, KEXTs and required files out of Mountain Lion, bless the volume and go? Well, you probably can. But you also need a license to do this. Also, if you do this, you've just basically made a copy of Mountain Lion. Isn't that sort of beside the point?

With Binary Cheats for generic hardware: As above (untested)

This pretty much is the OSx86 project. Again you pretty much run into legal issues straight away. There's no question that these last two methods are definitely possible – the fact that you can run Mountain Lion on generic hardware is proof of that – but the whole point of this was to be able to legitimately compile your own Darwin OS from source.

Side Note

You may have noticed I've deliberately avoided anything 32-bit. In a world where every major OS is available in 64-bit there's not much point in compiling a 32-bit one. Apple did actually provide disk images of Darwin (up until Darwin 9) here. They worked perfectly on my Windows box.

Concluding Remarks

I guess at the end of the day, people don't buy Mac for Darwin, they buy Mac for Aqua. As a result, support for Darwin as a standalone, open-source product has gradually declined to the point where it is really just a token gesture to the open-source community. The other slightly ironic fact is that in order to learn about a lot of this, you have to jump right into the OSx86 project, which isn't exactly sanctioned (to put it blandly). Even then, there's not a great deal of information around. PureDarwin is an excellent place to start, and Jonathan Levin's book is an invaluable reference for all things xnu.

This has been an extremely educational year of work, and I'm almost as happy knowing how to do it as I would be actually doing it. I'm going to have to stop work on it at some stage though and now is that time. As a final futile shout out to Apple, would it be too much to ask to have just one more build of Darwin when you release Mavericks?

Tags:

Linux

Kernel

Darwin

Recent Posts