Is staring into the cover sheet of the exam paper and looking at the second page considered cheating?

I don't know if I'm simply overthinking this or if it is a genuine ethical issue.

You're overthinking this. You are complying with all instructions, and I presume are not even intentionally exploiting any loopholes. To the extent that simply looking at the cover page allows you to guess the subject of the first question, that is a defect (if it can even be called that) of the exam.

A good exercise to test for ethics is to consider what the reaction be if you were publicly known to have "stared at" the cover sheet. Can you imagine anyone at a disciplinary board doing anything other than laughing at the triviality of the "offense"?

Whether you should attempt to avoid seeing through the cover sheet or whether it is "fair game" is perhaps a more complex question -- however, I suggest you find more significant matters to spend your mental energies on.

It's a matter of spirit and letter of the instructions.

According to the letter of instructions, you are not supposed to open the first page before instructed. This, you follow.

According to the spirit of the instructions your prof wants to have a unique starting time for the exam across the students being examined. This, you violate (or try to) by intentionally trying to see what the questions are.

Now, if you chance to see something once, it is not your fault if you utilise this, as an exception. You can not unknow. But you are basically taking explicit advantage of the weakness in the exam - you essentially try to hack the system by prying through the sheet.

A good exercise to test for ethics is to ask what would happen if your technique becomes commonplace (for more information, see Kant's philosophy).

Of course, the response could be that the examiners really print one extra empty page to be placed between the cover sheet and the questions to prevent this. So, hundreds of extra empty sheets would be printed, just to prevent one of the many possible cheating strategies, yours.

Note that, once your strategy becomes commonplace, mere instructions not to peek will not help; you may be sufficiently "lawful" to follow explicit instructions, but others may not be - the intermediate sheet will become unavoidable.

Your hack has affected (here, in a wasteful way) how exams are being carried out in the future. Note that your hack has in no way demonstrated your ability to cope with the material.

It would be different if, for instance, you would just find an extreme shortcut to solve a problem.

Another, much more interesting ethical question would arise if, say, for an exam about cybersecurity, a candidate would break into the lecturer's secure computer and steal the answers - after all, they demonstrated proficiency in the topic of the lecture, even if not precisely according to the marking scheme.

Yeah, it's cheating.

Whether you slide the page or not is irrelevant. You are deliberately accessing extra information. It's not inadvertent since you stare hard and long to try to read through and get extra info early.

Might as well lean back in your chair to peek at cards. Or look at someone else's test while they take it. Doesn't matter if the steps to prevent it are not ironclad. You know you aren't supposed to look at the questions early. But you do it. You are starting the test early by looking at the questions early.

Blatant cheat. At VMI, they would drum you out (literally). Heck, trying to justify it would add equivocation on top.