Best way to get into Kernel programming?

Firstly:

For the baby stages, writing various variations on "hello world" modules, and virtual hardware drivers, are the best way to start (real hardware introduces real world problems best faced when you have more of an idea what you are doing).

"Linux Device Drivers" is an excellent book and well worth starting with: http://lwn.net/Kernel/LDD3/

LDD (used to, at least) have exercises where you wrote virtual drivers, e.g. RAM disks, and virtual network devices.

Secondly:

subscribe to https://lkml.org/ or to the mailing list of a sub-system you will be hacking in. Lurk for a bit, scanning over threads, reading code review (replies to patches) to see what kind of things people stumble on or pick up on.

See if you can obtain (cheap) hardware for a device that is not yet supported, or not yet supported well. Good candidates are cheap-ish USB NICs or similar, low-cost USB peripherals. Something with an out-of-date, or out-of-tree driver, perhaps vendor written, perhaps for 2.4.x, is ideal, since you can start with something that works (sort-of), and gradually adapt it/rewrite it, testing as you go. My first driver attempt was for a Davicom DM9601 USB NIC. There was a 2.4-series vendor-written kernel driver that I slowly adapted to 2.6. (Note: the driver in mainline is not my driver, in the end someone else wrote one from scratch).

Another good way in is to look at the Kernel Newbies site, specifically the "kernel janitors" todo: http://kernelnewbies.org/KernelJanitors/Todo This is a list of tasks that a beginner should be able to tackle.

There are many kernels in existence.

Linux is not the only game out there, but it's the most widely used. Some people recommend the minix kernel to understand operating system concepts. I would go with Linux 2.6.x series as it's the most common and it won't be wasted on something (almost) nobody uses. Solaris, XNU (Mac), FreeBSD and Dragonfly are notable.1

Find something super simple (serial port, usb, etc.) and make it do something else. For an extra challenge, try writing a kernel module for something that doesn't yet have one. (Be sure to google the suspect by pci sudo lspci -nn and/or usb id sudo lsusb first.)

Update: Don't do this directly on your primary system, use virtual machines (VM) (computer in a computer) as mentioned. VirtualBox can provide this and it's free. Also, I don't want it to seem overwhelming, but try VM snapshots so you can go back in case something breaks the VM itself.

Update 2: The note about Git (DVCSes inc. mercurial, fossil, bazaar, darcs) is a good idea, mostly for code and small, important files. Try not to use Git to backup music, videos and such unless you know the implications. ;)

Find a module or driver you want to modify or start with a null / hello world type of module and build from there. Specifically work with a module because it will allow you to load, unload, recompile and reload the code into a running kernel. Later you can get into the core bits, but basically modules have all kinds of and you can do pretty much whatever you like.

Second, use a VM. Setup a guest machine under VirtualBox or some over VM and do your kernel testing in there for a while. That way you won't kill your main system when you break something, and restarting, backtracking to a working snapshot, etc is really easy.

Third, learn git.