How do successful undergraduate and PhD students differ?
I'm not a big believer in "types" of people. But yes, some succeed in an undergraduate program but don't in a doctoral program. The difference, I think, is largely determined by the nature of the programs, not the people.
As an undergraduate you are mostly exploring what is already known and the connections that are already known between them. Too often the method of evaluation is poor. Poorly designed multiple choice questions, for example, test memory primarily. Memory is a useful skill, but more is needed to complete a PhD. An undergraduate degree, even when focused on a particular field, as in UK, is still pretty broad.
A doctoral program, on the other hand, is an exploration of the unknown. You can't plan for success in research, only work effectively so that you can find it eventually. But it is much more frustrating for many people than is the exploration of the known. You are trying, among other things, to make connections that others have not yet made. It is a different kind of thing. The boundary between the two isn't smooth.
So, the undergraduate degree is only a partial preparation for what needs to happen in a doctoral program. Being successful in the one doesn't guarantee success in the other. It gives a basis on which to work, but that is all. Some people learn, too late, that they just don't like researching the unknown, even though they love what they did as undergraduates.
In mathematics, for example, if you are studying things already known and come to a hard part, you can find someone, or a book, say, to explain it to you. You can find exercises to deepen your knowledge. But as a mathematician researching some area, there are no such resources. You are a creator, not a consumer. There is no guarantee that you will ever succeed in answering some particular question.
So, I attribute the effect you see more to the different nature of the two levels of learning than to the "types" of students who succeed or fail.
However, as an afterthought, I think that most people will start out in their education finding things fairly easy to do. The things are fairly simple, of course. But usually people reach a point where it isn't easy anymore and they have to work to continue. I think that the people who don't reach this point early in an undergraduate program, or at least by its completion, will have a hard time in a doctoral program. If you've never had to work hard to advance, research can be a shocking awakening. I was lucky enough to learn my limits in secondary school and so learned to work hard at learning. Others, who far surpassed me early on, didn't advance as far. I doubt that this is a universal, but think it is pretty common.
I think the key difference is the openness in a phd versus undergrad. If you receive say a homework problem as an undergrad you can savely assume (in most cases) that this is a problem that can be solved in a reasonable amount of time with the knowledge of the course and its prerequisites. If you study a problem as a phd it might happen that
- the problem is still decades away from being solvable
- this problem was solved under a different name somewhere else
- once you properly understand it, it is actually trivial
- the problem doesn't have a good solution in any reasonable way
Finding problems that look interesting is not that hard but knowing which ones actually lead to a fruitful phd is hard and a problem that doesn't really occur as a undergrad. Your advisor will help but this is difficult for him/her as well.
I'll say this from the perspective of an undergraduate student where in my first two years my gpa was 2.7 and my last two were 3.5 which allowed me to get into graduate school and finish my courses with a 3.8. I am an MSc student, but after about 8 months in research I have had time to think about this question.
Undergraduate studies are tricky to think about, because what is the point of them, really? The institution needs to consider that a small fraction of people will pursue PhDs and so course work needs to be tailored to many more people. So the question becomes, how should you approach your studies if you want to be a researcher as opposed to how to be "successful".
Looking at it from the lens of undergraduate studies being preparation for a PhD, you are essentially learning a lot of different tools and techniques. Consider any sport, when you're training at the lower levels you are trying to grasp onto basics and principles, but what separates young (undergraduate) and elite (PhD) athletes is how the elite athletes use those basic skills and principles to pull off incredible highlights. Elite athletes don't think about how to dribble the ball or move in a certain way, instead, those basic principles are ingrained into them so they don't have to think about them at all. As the undergraduate, it is your job to gain mastery over basics so that when you try to break into your PhD you're able to use those fundamentals in new and creative ways. As the undergraduate you answer homework problems you've never seen before, now you're job as a PhD is, in a way, to be the one to write those new problems. When I started to see a boost in my GPA it was not because I suddenly "got it", but because instead of just studying the material, I would create my own tests and try to think of all the different ways I could play with the material. I begun to take those basics and create "new" things with them. By doing this, you are essentially expressing concepts you're familiar with but in new ways. For example, I would write down my own integration problems and try to solve them with the techniques that I had - sometimes what I wrote was analytically solvable, sometimes it was not.
As a PhD you're now primarily focused on finding new problems in the literature, and this is really challenging and a skill that I believe can be developed and nurtured. Because I believe that problem finding is a skill that can be developed, the notion of creativity instead becomes how well you can connect the dots that are in front of you (for most of us). But again, how can you connect the dots if you lack the skills to be able to draw the line between them?
As a PhD you are flexing brain muscles that you never really had to before, so it's frustrating at first but I believe over time it gets easier as you adapt to that type of new thinking. Research, like homework, is practice and the only way that you will get better at being a researcher is to practice doing it. So don't let yourself give up before you've had the chance to start.