Chemistry - How is nomenclature decided in this case (formyl vs oxy)?

According to the new recommendations of IUPAC nomenclature rules, when choosing the senior parent structure, you have to first consider the senior parent structure, which has the maximum number of substituents corresponding to the principal characteristic group or senior parent hydride in accord with the seniority of classes (functional groups). Then, if there is a choice, choose the principal chain, which has the greater number of skeletal atoms. Next in order is the chain, which has the greater number of multiple bonds, and then has the greater number of double bonds (This is according to Blue Book in 2013, which was displayed in 5 steps you have mentioned from Wikipedia).

Accordingly, your most senior function is carboxylic group, which would have the highest priority:

Then, there is a choice to choose the principal chain, which has the greater number of skeletal atoms. Both choice have six carbons, but the numbered chain indicated in the image has additional function, thus it is the principle chain. You also need to consider the stereochemistry if they are indicated. Here, there is a double bond in trans-orientation. That should be included in the name. Although, $\ce{C}$-5 is a chiral carbon, its stereochemistry is not indicated so you can leave it alone. Therefore the correct IUPAC name is: (2E)-2,5-dimethyl-6-oxohex-2-enoic acid.