How to find balance between overconfidence and lacking confidence?
1) How should I view the interaction? Should I assume I am probably wrong? Should I assume that he is wrong? Another option...?
Researchers should have differing opinions on topics such as whether non-explainable machine learning techniques should be used in healthcare.
2) In the future, should I press the debate more rather than only asking a few questions and thanking him for the info? Would this be impolite at a conference? Would it be an unwise career move?
Such opinions should be debated, that's how research advances, and such debate should be encouraged, at conferences and elsewhere. Some researchers will be less willing to accept your opinion than others, only you can decide whether you want to engage with such researchers.
Should I raise my disagreement when this occurs?
If you want to.
How long should I debate the issue if we don't agree?
As long as the conversation lasts.
If we don't agree in the end should I assume I'm wrong?
No. There doesn't seem to be any right and wrong here, just opinions.
xedg, there seems to be two issues here which seems to complicate your mixed feelings. One issue is around status, which you alluded to, with your first author publications and strong academic record. The second issue is around where is the best environment to debate and clarify the pros and cons around the role of machine learning.
The first issue around status is quite unfair as you positioned the professor as being disrespectful or "dismissive". First conference presentations are usually always given great courtesy especially in question time. So what you interpreted as dismissive, may possibly be respectful distance. You are also not in a PhD program or have not finished your PhD, so experts will always back off and not put you in difficult spots out of respect of your position despite your accomplished publication record - common conference etiquette blog Remember having a bad experience at a conference will look bad for the conference and may also discourage people who may want to do a PhD. Attacking junior researchers will also diminish the professor in the eyes of other senior experts. Making junior presenters cry has been a subject of discussion among experts and does not look good at all. So all these factors may explain why debating an undergrad is poor form at a conference question time post-presentation. The other possible issue is around familiarity. As you attend more conferences and become a familiar face, people will be far more comfortable debating and clarifying complex issues with you compared to your first conference.
The second issue is what is the most appropriate format for this debate. If you want to flesh out "the role of machine learning in healthcare" at a conference, the short question time post-presentation is definitely not an appropriate timeslot for such a complex discussion. Symposiums or panel discussions are far more a satisfying format with the appropriate adjudication and moderation for this to occur. A senior mathematician would have a very different perspective than a clinician for example, but in a panel format, all the views would be respected rather than just your narrow focus on what is "right or correct" as implied in your question.
A review article examining the pros and cons of symbolic logic AI as the dominant approach in healthcare then be more appropriate in an extended article, rather than at a conference? To expect an expert to synthesize his academic focus and all the publications in a field is unproductive and not representative in a short question timeslot post-presentation.
I liked this Indian Hills presentation on how not to be argumentative might be helpful. Although you might not be overly argumentative, it always pays to move away from the "correct vs wrong" mentality in the academic realm. Most things are "more right" and "more wrong" depending on the scenario and circumstance.