Should I email my professor to clear up a (possibly very irrelevant) awkward misunderstanding?

If I were you, feeling uneasy about your previous conversation, I'd stop by during your professor's office hours rather than sending an email to express your concern. It's more personal. Pick a time when he appears free. Tell him you've been feeling bad that you may have given him the wrong impression, that you want to make clear that you really like the class and really did find the readings helpful. It's pretty likely he'll set your mind at ease, reassuring you that he didn't take offense and wasn't upset but appreciates your stopping by to discuss.

I would certainly talk to him. It's maybe not necessary to rehash what was said in the exam, that could be awkward. But it's worth asking for an appointment to discuss your research interests and potential thesis advisors. During that conversation, you can "follow your nose" to see if he really isn't a good fit for you as advisor (e.g., because he has too many students already) or if it's something he is open to revisiting.