Chemistry - Which is the more stable enol form?
Solution 1:
Keto-enol equilibria reflect a delicate thermodynamic balance between the two forms.
The bonding differences between the keto and enol structures shown above are:
keto: C=O double bond, C-C single bond, C-H bond
enol: C=C double bond, C-O single bond, O-H bond
If we look up the bond energies for these bonds we find
keto: 745 + 347 + 413 = 1505 kJ/mol
enol: 614 + 358 + 467 = 1439 kJ/mol
This rough calculation tells us that, generally, the keto form will predominate. However, the energy difference between the two forms is only 66 kJ/mol, a relatively small number; so it is likely that small differences can cause a dramatic shift in the relative concentrations of the two species.
Some general guidelines for predicting shifts in keto-enol equilibria
Aromaticity and conjugation - The keto-enol equilibrium for phenol (I) lies entirely on the enol side due to thermodynamic stabilization provided by aromaticity. Compound III would be expected to have a higher enol content than compound II due to the extended conjugation present in the enol of III
Substitution - Replacing hydrogen with alkyl groups on a double bond stabilizes the double bond. Therefore, we would expect compound V to have a higher enol content than compound IV
Dipolar repulsion - The carbonyl dipoles in butane-2,3-dione (VI) can reduce their electrostatic repulsion by adopting the geometry pictured with the carbonyl groups oriented away from one another. Due to the constraints of the 5-membered ring, cyclopentane-1,2-dione (VII) cannot adopt a similar geometry. In this case, the dipolar repulsion is lessened by increasing the enol content in the equilibrium to a point where the enol predominates.
Hydrogen Bonding - Hydrogen bonding can stabilize the enol form. If the hydrogen bond is strong enough, and particularly if other factors also stabilize the enol form, the enol form can predominate. Your molecule, 2,4-pentanedione (IX), is a good example. There are two possible enol forms, VIII and X. X is by far the predominant enol as it has 1) extended conjugation and 2) substitution on the carbon-carbon double bond. In addition, X can form a very stable hydrogen bond involving a 6-membered structure between the two oxygen atoms.
Solvent Effects - Especially in cases where hydrogen bonding is involved in stabilizing the enol, the solvent can have a dramatic effect. In benzene, where intramolecular hydrogen bonding predominates, the IX:X ratio is approximately 5:95. In water, where the intramolecular hydrogen bond is replaced by hydrogen bonding to the solvent, the ratio is roughly reversed.
Answer to your question
Cyclopent-2-enone can form two different enols, XI and XIII. Neither enol has any features that would be expected to provide significant enol stabilization. There is no more conjugation than what is found in the ketone, there is no special hydrogen bonding, and no substitution of the carbon-carbon double bonds. Indeed, the proton nmr of cyclopentenone is just as expected, no evidence for a significant amount of enol. On the other hand, 2,4-pentanedione exists primarily as the enol form, at least in non-polar solvents, for the 3 reasons (conjugation, hydrogen bond, double bond substitution) discussed above.
As an aside, you mentioned that a 5-membered ring with two double bonds (a cyclopentadiene) might not be very stable; actually, they are quite stable and quite common, 2 double bonds in a 5-membered ring do not create a lot of strain.
Solution 2:
The 2,4-pentanedione enol form shown is the more stable enol, since the lone pair on the enol oxygen can delocalize across the five atoms to the electronegative carbonyl oxygen. Hydrogen bonding is also a minor factor in its favor. The enol form of cyclopent-2,3-en-1-one is indeed (also) a conjugated system, but the lone pair on the enol oxygen cannot delocalize to an electronegative atom, rather only to a carbon atom, and the delocalization occurs only over three atoms instead of five. In thinking about this problem, it may help to draw out and look at the various "resonance structures" of these enols, but from a molecular orbital standpoint, the governing principle is that a more delocalized orbital is lower in energy (when it contains electrons, that is).
The effect is more pronounced in comparing the enolate anions of the two species.
Solution 3:
Due to interamolecular hydrogen bonding Enol form in this case is more stable