Embedded Development - Taking the Next Step
So from the previously linked answer, I think the most interesting/beneficial things for me would be the bullet about learning the tools (compiler and linker), and learning different styles of software architecture (going from interrupt based control loops to schedulers and RTOSes)
Porting a small operating system to a new device could help you to understand schedulers and RTOSs. FreeRTOS is popular and well documented. eCos is another.
Writing a bootloader is a good way to get to grips with a linker as you'll want to divide up memory and flash into regions.
Another tip is to pick a completely new architecture or chip and build yourself a development board. Forcing yourself to start right from the beginning and look everything up in the datasheet is a good way to learn.
Explore Protothreads. Try writing the same programs in both a threaded and state machine style. Once you're done with Protothreads, write a real thread scheduler.
The MicroC OS II book is probably something to invest in. You should also create projects to learn the various interfaces i2c, spi, mdio, etc. In particular how to bit bang each one. From time to time the hardware will support the bus (need to learn that on a vendor by vendor basis) but often for various reasons you wont be able to use i2c/spi hardware and have to bit bang.
The avr/arduino is fine, you should learn ARM, thumb and thumb2, and the msp430 and some older (non-mips) pic. Look at the bootloader code for the arduino, and figure out how to make a loader program, erase the flash on it and take over the board/chip. Get an lpc based arm micro, same deal look at the serial port programming protocol. Get a sam7s or something with an arm7 that has a traditional jtag, get an olimex wiggler or jtag-tiny (I recommend the latter). Get comfortable with openocd. ARM's swd is more painful that normal jtag but in this market will prevail (for cortex-m based products). In short, learn the various ways that vendors provide for in circuit programming. You will brick boards from time to time and will want to be comfortable with unbricking them. Along these lines, write some code to parse intel hex, srec, and elf files, you may someday need to write a loader and need to support one or more of these popular formats.
You mentioned tools. You cant go wrong learning gcc and binutils, learn how to cross compile, at least for the supported platforms (usually involves --target=msp430 --prefix=/something for example). Supported platforms for the mainline gcc and binutils is a moving target so avrgcc and mspgcc and the like are basically done for you. You need to learn to write linker scripts, how to write your C code so that for example fixed tables show up in the rom not the ram. Also get a feel for disassembling the binaries, you need to insure that the tables are in the right place, insure that code is where you think it is, that the vector tables and boot/startup code are where the processor needs it to be to boot. Also doesnt hurt to find out what the compiler optimizations do and what C code looks like when compiled to assembler/machine code. if possible dont limit yourself to gcc/gnu. llvm is a strong player, it has the potential for passing gcc by as a better tool. You may have already used sdcc. try the eval versions of kiel, iar, etc. You will find quickly that there is a lot of grey area in the C/C++ standards and each compiler interprets those differently, also dramatic differences in the quality of the code produced from the same high level source. If you stick with this profession there will be times when you are forced to use a not so great compiler and have to work around its warts and weaknesses. In the desktop business you can often get away with refusing to use non-standards compliant tools. In the microcontroller world, sometimes you get what you get and that is it. Sometimes you get vendors that modify/enhance the C language to meet their hardware features or supposedly make your life easier (rabbit semi and xmos come to mind). (xmos is a very attractive platform for many reasons, I consider it advanced, but from the sounds of your experience you are likely ready, the tools are a free download, really good simulator, important to learn to study .vcd/waveforms of your code executing).
chibios is another one to look at.
Creating sucessful bootloaders is an important skill. The bootloader, or at least the beginning part of wants to be rock solid, you dont want to deliver a product that is easily bricked. A simple boot with a way to re-load the application portion of the flash without compromising the entry part of the bootloader is key.
The stellaris eval boards are loaded with peripherals, although they provide libraries it is worth learning, esp since how they tell you it works and how it actually works differ and you have to examine their code and other resources to find out.
Being an avr fan, if still out there, I recommend getting an avr butterfly. Learn a little serial programming, solder on a connector and reprogram it. A few peripherals on there to learn to program.
maybe get a formerly dallas semi, now maxim I think, one wire device. Like their temperature sensors. Even more painful than i2c and mdio with their bidirectional data bus, this one wire thing, is one wire (and ground). Power, master to dependent and dependent to master all on one wire.
When I was where you are now, I found decoding infrared remote control protocols fun. The ir reciver modules are easy to come by, radio shack actually had a good one. Basically the opposite of bit banging, you want to measure the time between state changes, using that timing detect and or decode the protocol. A universal receiver is not necessary, one protocol at a time is fine. Likewise being able to then bitbang commands to an ir led, in particular if you bit bang the carrier frequency.
Talking to an sd card via spi is probably a good idea as well. Definitely learn how to erase and program i2c and/or spi flash parts, you will come across these often for serial numbers and mac addresses and the like.
I recommend learning the basic ethernet protocols as well. Be able to parse and create arp, and udp packets (from scratch) (and icmp/ping as well). Pretty easy to create an udp stack if you cheat a little, not following the the actual arp rules, if someone sends you something send the reponse back to the sending mac/ip. Or perhaps go so far as to watch the arp packets for other folks go buy and keep track of the mac/ip addresses around you. tcp takes a lot more work, doable, but better to just read about it first than to try to implement it.
Good luck, and most important, have fun.
That's a big question. Unfortunately there is no one way to learn embedded microcontroller development, just like with learning most everything else. I'll assume that since you're here asking, you don't just want to get a specific project done but want to really understand what is going on so you can get any project done yourself later. That's a good start.
There is no substitute for doing a few projects in assembler to really understand things. Part of the job of high level languages is to hide some of the complexity, but that also obscures things you want to understand. Arduino is yet another layer up from that. It may be fine for someone that doesn't want to know how things work to get something done, but it's really bad if the objective is to learn the details. So ditch the arduino, forget you have a compiler, roll up your sleaves and get ready to really dive in.
The first place to start with any microcontroller is with its datasheet. The various micrcontroller lines have different details, but the concepts are the same. Pick one and stick with it until you've done a few projects and feel comfortable with it. The TI MSP430, Atmel, and Microchip PICs are all mainstream microcontroller lines. I'm a PIC guy so I'll talk about that. It's not wrong to go with one of the others, but I think there is more accessible community support for PICs out there, and the PIC line is very broad so it will be easier to go to both larger and smaller micros from where you start.
If you start with the PIC line, I recommend the 18F2520. It's quite a capable microcontroller, comes in a easy-to-screw-around-with 28 pin package, has a decent amount of program memory and RAM, and a decent set of common peripherals. Get a small handful and use it to start with even for small projects that could have been done with something less. The point is to get to know it well. Eventually you'll want to do something that requires more than that PIC can do. Hopefully by that time you'll be pretty good at the basic stuff and can easily substitute a different model. Actually, all 18F PICs are the same except for the amount of pins, program memory, RAM, and the peripheral mix. After a few projects with the 18F2520, you'll have no problem jumping right into any PIC 18.
If you end up doing this professionally, eventually you'll run accross a requirements that dictate something smaller or cheaper. That's when you look at the 16F or possibly even 12F or 10F families. Things will look largely familiar, but a few things will be a little more difficult. This should be no problem to handle if you're starting with a solid understanding of a 18F. Nowadays, the 16F and lower families are really only for volume production or perhaps when physical space and sometimes power are very important. There is little advantage in them compared to a 18F for a hobbyist.
Going up, you can look at the 24, 30, and 33 families. These are basically all the same except for some minor wrinkles like what voltage they run at, top speed, and whether they have the DSP math ALU or not. They are more complicated to program in assembler, but also strangely easier. There are more instruction to keep in mind, but its therefore also easier to do a lot of things. This is also a good point to start getting back to C if that's where you want to go. The C compiler for these parts is actually quite good, unlike the C compilers for the PIC 18 and below. It's also easier to mix C and assembly in the same project on these parts. Don't forget assembler altogether for these though. It will still be useful to know when you have to write a bootloader, deal with multitasking which performs unnatural acts on the stack, write really tight code to read a bunch of A/Ds and compute the PWM duty cycle for the next pulse of your 100KHz PFC power supply, etc.
One nice thing about the Microchip line is that a single programmer/debugger works accross the whole lot. If you're serious about doing this professionally, get the Real Ice. To save a little money, you can get the ICD3, which is just a slightly stripped down Real Ice. The one thing it's not is a ICE (dumb marketing strikes again), but it is a good in-circuit debugger. MPLAB, their IDE debugger and simulator, also works accross the whole product line.
Again, for professionals or those wanting to become professionals at this, go to the annual Microchip Masters conference. Yes it's in Arizona in the summer, but you're there to learn, so forget about the 115 degF in the shade. (Actually I like exploring Arizona in the summer, so I go out a week early with my little tent and bump around the hinterlands. It's amazing how few people are out in the vast national forests only a short distance from Phoenix.) Masters is also a good place to pick up development tools. They usually sell them for something like 45% off at Masters. (I'm a platinum level Microchip design partner, so I get them 45% off all the time, which is why I'm not up on the exact Masters discount).
If you start exploring microcontrollers by doing PIC projects, check out my web site where I get into more details of how I usually structure PIC projects and give you a bunch of free tools to do some fancy things. If you understand good software design and cleanliness, you'll get aggrevated at MPASM. Do a project or two to understand what it does and doesn't, then look at my wrappers and other tools that let you do a lot more while sticking to good software design practises. Assembler doesn't have to be a mess just because most of it is. If you're really into it, you might stop by the class I'm teaching at Masters this summer on this subject.