32 kHz crystal not operating as expected
There are two main things going on:
- You don't have enough load capacitance.
- You don't understand load capacitance.
Imagine one side of a crystal being driven with a sine wave at the crystal frequency. This signal is low impedance. The load capacitance is that capacitance you put on the other side of the crystal to cause a 180° phase shift.
The phase shift of such crystals varies quickly as a function of the frequency at the crystals operating frequency. Since the phase as a function of frequency is very steep right at the operating frequency, this is a good thing for the driving circuit to use to ensure the crystal is operating at the intended frequency. These types of circuits oscillate optimally when the crystal shifts the input phase by 180°. Since only a little frequency change messes that up, the resulting oscillation is very close to the crystal's intended frequency.
Now back to your circuit. The big clue is that things work when you put a scope probe on the oscillator input pin. What that is doing is adding capacitance on the output side of the crystal. Apparently, with the setup you have, the additional scope probe capacitance causes the crystal to shift the phase the appropriate amount for the system to oscillate. If you add more capacitance yourself to the crystal output only, you replicate the effect of the scope probe and things will work. Try another 10 pF or so for starters.
Don't use formulas you find on the other end of the internet without understanding them. The equation you show makes a bunch of assumption, some of them not valid. Unfortunately there is a lot of conventional stupidity out there regarding crystals.
The crystal by itself is only a two-terminal device and doesn't "know" anything about your circuit ground. Ultimately, the load capacitance is what is across its terminals. The conventional stupidity therefore says to use two equal capacitors on each side of the crystal to ground. Since these are in series, each need to be twice the desired capacitance. However, whatever stray capacitance to ground you think there is on each side of the crystal needs to be subtracted from these capacitances.
The problem with the conventional stupidity is that it ignores the impedance of the crystal driver output. Consider the extreme case where that is 0. In that case, the capacitance added on the input side of the crystal is completely irrelevant, since it is in parallel with the 0 impedance of the driver. The load on the crystal is then only the capacitance on its output.
Do some math. The impedance of 6 pF at 32.8 kHz is 810 kΩ. Now the impedance of the crystal driver is certainly not zero, but quite likely significant relative to 810 kΩ.
Consider what each of the caps really do. The one on the input loads the crystal driver. The main purpose of that is to attenuate some of the harmonics coming out of the driver. This beats on the crystal less, and makes it less likely that the whole system will oscillate at a harmonic. Crytals have complex transfer characteristics. They can have some of the same characteristics at harmonics as they do at the intended operating frequency. Some crystals are cut so as to deliberately enable use at harmonics, called overtone mode in the industry.
The capacitance on the output is the true "load" capacitance. Its reactance works against that of the crystal to phase shift the result the right amount at the right frequency.
In your case, the crystal is rated for 6 pF load, and that's what you put on its output. That should have worked. My guess what is happening is that the cap on the input of the crystal, really on the output of the crystal driver, also caused a phase shift that worked against that of the load cap. Just as a test, try removing the cap on the crystal input and leave the 6 pF on its output. It would be nice to see the waveshape on the crystal input then, but even a 10x scope probe might change it. Try it anyway, but make sure the scope probe is set to the highest impedance, therefore the lowest capacitance, possible.
The MCU maker is likely at fault. There is absolutely no excuse for not designing a modern MCU RTC oscillator to reliably function with any typical commercially available 32kHz crystal.
Unfortunately, the opposite is much more common, as you have already discovered - in your case the MCU data sheet fails to mention that 6pF load capacitance does not work.
The root issue is that you are dealing with a system of two components, made by two different manufacturers. One of them speaks silicon and the other speaks quartz, and they have never properly agreed how to tell designers how their products reliably work together.
So, as you have found out, the crystal oscillator can be a trap for the unweary. I have seen a major automotive production line grind to a standstill because of crystal oscillator startup issues!
Anyways, to get to your question of WHY, there are four important parameters at stake:
Output impedance of the MCU oscillator. This varies over frequency and is often complicatated by configuration bits such as "drive level" or "power level". I have never seen these values specified/guaranteed by any MCU maker.
Input impedance of the external capacitor-crystal-capacitor "pi" network. This is primarily determined by the capacitor on the input side, which in turn is determined by the load capacitance specified by the crystal maker.
Voltage gain of the (loaded) MCU oscillator at resonance. The oscillator gain has to make up for the ESR-induced loss of the external Pi network. This gain changes significantly over temperature, supply voltage and manufacturing batch. I have never seen this gain (and driver output impedance) properly specified /tested/guaranteed by any CPU maker. Some makers specify transconductance \$G_m\$ instead of voltage gain.
Voltage gain (actually loss) of the external Cap-Xtal-Cap "Pi" circuit at resonance. This is primarily determined by the internal equivalent series resistance (ESR) of the crystal. The crystal you mentioned specifies ESR=50k. The resistance also increases over age (as moisture/impurties leak into the crystal case) and is also affected by soldering temperature/time. (Impurities in the crystal case evaporating & settling on the quartz) ESR can also vary significantly between manufacturing batches. 50k is a fairly typical ESR for a 32kHz crystal - the lowest I have seen specified at 32kHz for small form factor crystals is 30k.
For any oscillator to work, the total voltage gain, which is the the product of (3) and (4) must >1. In addition, the phase of the gain (yes, gain is a complex number) must be 360 degrees. About half of the phase, 180 degrees, is provided by the inverting amplifier, and the "second inversion" is provided by the cap-xtal-cap network.
Here's a simple online simulation that can help you get a feeling for how gain, output impedance and the capacitor values interact and affect startup. Right-click any component to change its value. (Note - this simulation uses 1mV residual capacitor voltage to fake startup, but in real life noise in the amplifier is the source of startup, like in this one)
So what happened in your case? Most likely, the MCU oscillator designer designed his output stage to reliably function with 12.5pF loaded crystals, and it turned out that at 6pF loading, either the voltage gain or phase requirements were simply not met. Since nothing about the design assumptions is stated in the data sheet, voila, problem for you - and many others.
Wow, what should an embedded designer do?
First, always be aware that a marginal crystal oscillator can cost your business a lot of money.
Second, in light of the above, especially if you lack experience or if your MCU vendor doesn't specify crystal parameters in the datasheet, your best investment could be an external low power 32kHz oscillator.
Third, ensure you use a crystal with ESR and capacitance specified by your MCU maker. If you don't see any in the data sheet, ask your supplier for a list of recommened crystal part numbers, or choose an MCU that does.
Fourth, test, test, test! Over all voltages and temperatures. Note how long startup takes by timing it in firmware using an RC clock if possible, and if production units exceed the norm by, say 2x, let your test firmware set a flag so it can be noticed in production testing. That way, production units can't get out the door with marginal oscillators without alarm bells ringing.
What do experienced production verification engineers do?
They work around the general lack of proper information by requiring a 10x safety margin between "what works" and "what works reliably" - they measure the actual ESR, then add an extra 10x additional "handicap resistance" in series with the crystal into the cap-xtal-cap network. If the "handicapped ESR" system works over all voltage and temperature combinations, then it's assumed that the 10x safety margin is sufficient to cover the unknown variabilities in both ESR and MCU gain. This is partially explained in figure 3 of this application note.
What should YOU do?
If you can't perform the above test for any reason and want to sell a product in thousands, you are certainly better off investing the extra pennies for an off-the-shelf 32kHz oscillator from an oscillator vendor who has done all that testing for you, or by switching to an MCU that specifies a specific crystal (or crystal requirements) in the device data sheet.
While you may "fix" the situation by selecting a crystal with lower internal resistance and/or by playing with different/asymetrical capacitor values, your solution could still be marginal, for the reasons explained above.
TL;DR:
Crystal oscillators can cost your business a lot of time and money. Use an external oscillator if you can, or peform the "handicapped ESR" test as described above over all voltage and temperature ranges.
Finally, be sure to use NPO capacitors for temperature stability.
The two caps and crystal operate as a 180 degree phase shift. The magnitudes of the two caps (ratio) will define the voltage transfer ratio. 6Pf sounds a little small, the issue is what is the crystal parallel load design point? You don't want to move far from this value. I typically have 27pf on each side.
I also see one sode if the crystal tied directly to the processor output. This output might be low Z which which can over drive a crystal. Remember this drive spec of these watch crystals is tiny, very easy to over drive. A series R of 100K can be used to reduce crystal drive.
Make sure the processor has in internal 1 - 10 Meg bias resistor from output to input. You mentioned it starts to oscillate when touched with a scope prob. That might be a DC bias issue (10Meg scope probe I guess) or probably the probe cap adjusting the tuned circuit transfer ratio.
Be really clean (no stray flux) and really short wires. It is a real Hi Z circuit.
Bob K.
Also: The "Standard" probes I use are x 100 as they provide the smallest amount of capacitance, I recall about 1.5pf. Using x 10 is difficult on this circuit, x 1 is useless. Ues x 100 and crank up the scope vertical gain, make the scope front end do its work. X 1 probes are almost useless for high Z or high speed. You will love the x 100 doing digital works as the GND clip currents are down by a factor of 10. Try It.