How can I tell if I have simplified my talk too much?

I definitely hear where you are coming from.

In my modest opinion, a good conference talk is not necessarily the one that lets me understand the technical contribution (I can read the paper for that), but the one that lets me understand

• The importance and/or hardness of the problem studied
• What were the key new ideas
• Why they worked and when can I expect them to work

If I got this from the talk, I may be able to recall the result or the technique in the future when I have a need for it.

Of course I am not saying there should be no technical detail, the key idea may be the way a theorem was proved, but I believe any technical content should not be gratuitous.

It's important to keep in mind that different talks have different audiences and goals. How much you should simplify depends heavily on what kind of talk it is:

• For a job talk, part of the goal is to convince the audience that your research is advanced and hard. Oversimplifying is very dangerous in this context - frankly, it's better if parts of the audience does not understand your talk than if everybody understands it and some people think it's very easy. Simplify the problem and the conclusions (this is the part that everybody should get - why do you do this and why does it matter?), but don't simplify the actual technical content.
• For a public outreach talk, i.e., any talk directed at a general audience, it's important that everybody can follow the talk, while they won't care in the slightest if your methods are really complicated. Simplify as much as you have to for your audience to follow (how much this is clearly depends on who exactly the audience is - at a developer conference not much simplification may be necessary, when you are talking at a "Science for Kids" event you will need to break it down considerably).
• For a conference talk, opinions vary. I have come to the same conclusion as Pronte (that a conference talk is ultimately more an advertisement for reading the paper), but not everybody shares this sentiment. Decide what you want to do with your talk, and simplify enough to achieve this goal.

Another angle of this discussion is also which parts you should simplify for which audiences. For instance, I do a lot of empirical work nowadays (e.g., interview or survey studies). Oftentimes, the subject of my studies are technically deep topics (so the subject itself is rather complex), and the methods are, well, not exactly complex, but they are often lengthy to explain properly and somewhat intricate. How much I go into the details of either of these two aspects strongly depends on who I give the talk to. A scientific audience may appreciate me talking for 5 full minutes about how subjects were sampled and how results were coded etc, but an industrial audience will not care - "we did a study and interviewed X developers" is all the detail they need to hear. Conversely, the scientific audience is often not particularly interested in the technical complexities of the study subject, but an industrial audience almost certainly is.

I think that in almost all cases, people go to talks for insight not for details. If there is a reference/citation of a paper (or several) that provide the details, then the details needn't be in the talk itself. If you are at a conference with 300 listeners for your talk, then only half a dozen or so are likely to be interested in the pedantic details. But all are looking for the essential insights.

Some are also looking for ideas that they can expand themselves, of course. Students looking for important trends and interesting research possibilities. Even they don't need a complete, finished, product to make an informed decision. Insight triumphs.

But even in a formal lecture to students, which is expected to be quite detailed, you still expect the students to either have read the material before the lecture or to follow it up with readings and exercises afterwards.

The talk should say something about why the issues are important and about what the key, insight expanding, results were. It doesn't need to recapitulate the entire research process to do that.

If everyone understands it at some level it is a good thing. If a few want to follow up and go deeper it is also a good thing. Exhaust the audience with insight, not details.

As simple as possible, but not simpler.