Could one become a successful researcher by writing some really good papers while being outside academia?
My impression is that the likelihood of this is proportional to how theoretical the field that you have in mind is. In math or theoretical physics, it's at least imaginable for an outsider to produce an important new discovery. In High Energy Physics the idea alone seems outlandish.
That said, even in math the ratio of cranks (deluded amateurs who are convinced they have made an important discovery without having actually done so) to actual amateur geniuses is almost infinite to one. If you want to do research, in any field, the winning strategy is to study under a respected member of the community and learn how to produce, evaluate, and communicate your ideas in this way.
There is nothing that prevents you from doing so. It's just very hard for a couple of reasons:
- Without working at an academic institution (or something similar), you lack the environment to exchange and work on ideas with peers.
- Similarly, you need to discuss recently published results with peers to develop timely and relevant research directions.
- Every field of research has a certain publication culture. Without any advice on it, publishing is much more difficult. While you could get help from elsewhere, it's certainly a lot easier to get it from your advisor if you have one. If your results are obviously break-through and you know roughly how to write understandable scientific documents, this may not be so relevant, though.
- In case of experimental research, you often do not have access to the necessary laboratories and its equipment.
- Funding may also be an issue. In the experimental sciences, you may need to buy equipment or consumables for experiments. As another example, in computer science, you may want to publish at conferences, which comes with participation fees and travel costs.
- A good institution name can help to get you in contact with other researchers more easily.
- Access to publications is often easier as many universities have subscriptions.
There are probably many other reasons which I forgot.
Bell Labs researchers have won, collectively, nine Nobel prizes for work done outside of academia.
If a Nobel prize winner doesn't count as a "successful researcher", I don't know what does.