Doing research in academia and not liking competition
Actually, the competition is largely in your head, not in academia itself.
I'll never be able to win a stage of le Tour de France, so why should I ride a bicycle? I'll never win the final at Wimbledon, so why should I play tennis?
Of course, there are extremely competitive corners in academia. If you are in a "hot" research area where many many people are chasing exactly the same very few goals, then, yes, you are likely to get scooped.
But imagine two scenarios.
The first is that you have a thousand people at one end of a field and there is a single prize at the other end. Everyone runs to get that prize but only one can succeed.
The second scenario is that the thousand people are wandering around the field, each seeking something that they find interesting. Here, everyone can succeed.
Academia is, except in a few instances, much more like the second scenario than the first. Collaboration is possible. Two can enjoy a sunset. But only one can capture the flag.
Another competitive scenario is being one of many junior faculty at a very (very) top university, in which only one can be promoted to a tenured position. It is, of course, very competitive and collaboration with your competitors may be sub-optimal. But collaboration, even here, with others is not to be spurned. Even being second or third on an important paper is a good thing for a beginning academic, so long as you don't quit with just that.
But most universities, even very good ones in the US, aren't like that at all. Life can be good. But there are also some people who thrive in such a high pressure environment and would have nothing else.
My experience in academia was that the greatest thing was that I could think my own thoughts and pursue my own goals. Much of that was in collaboration with people. Some of those folks were just about like me, and some were internationally known superstars. But it was always fun.
I studied at R1 universities, but taught there only briefly (visitor). But my sense of it was that even for people in the same narrow field, collaboration was highly valued. The most senior professors, were happy to share ideas with junior faculty in field-centric seminars. Often those junior faculty (and we grad students) would develop those ideas, even with help of the top researcher. It was a shared process to extend what was known.
Of course, if a person has a lot of ideas, it is also often the case that they don't have time to completely explore them. For such people, generosity in sharing those ideas costs them nothing. They may not be a co-author of every paper, but their stature in academia rises nevertheless.
Don't think of academic, or research in general, as a zero-sum game. Everyone can win, especially if everyone has their own goals and are not somehow driven to adopt the goal/value system of others.
The field of research is broad and richly endowed. Find the bits that are interesting to you.
I'm not sure what these wishy-washy answers are trying to accomplish. Make their authors and you feel good? Try to lure you in academia because it's a cult and the more people try to join, the better? Working in academia isn't riding your bike every other weekend because it's fun. It's a job. It's training with your bike every day and watching videos at night about biking techniques, because if you're not in the top 10% at the biking competition, then you won't even get to enter the next one.
Academia is a competition from start to end. You have to perform well in school to get a PhD stipend, then you have to perform well during your PhD to get a postdoc, then you have to perform well during your postdocs to get a tenure-track position, then you have to perform well to get tenure, then you have to perform well to get recognition, grants, promotions... At each stage the funnel becomes smaller. If you get the thing, someone else won't, and vice-versa.
Don't get me wrong: it's not a dog-eat-dog world out there. You will collaborate, you will make friends, you will meet mentors who will help you through your career... But this doesn't change the core fact that yes, academia is a competition, and yes, it puts a tremendous amount of pressure on you. There are not many jobs out there where the line between personal and professional life is so blurred, and they usually pay much better. You will be required to perform at your best all the time, and if you don't, this will visibly and immediately stunt your career. You don't have to be the very absolute best at what you do to succeed in academia, and you will always have peer who are much faster and must better than you; but if you are not among the best – I'm not going to be able to give a precise number here – then you will not even get a choice about whether this life is for you or not: you will be kicked out. You will be able to do research and enjoy yourself for a while, sure, but at some point, time will catch up with you.
I'm not trying to scare you, but at the same time, I don't see the point in sugarcoating the truth.
You have a rather big misconception if you assume and experience that academia or research is mostly consisting of competition. Maybe even amplified by reading too long on this site where many Q&A's are about "being first author" and publish as much as possible. I also guess you are doing research in an engineering field where research is more about improving incrementally figures of merit, not real fundamental research, which is mostly about complementary research questions among groups, everybody doing the same would be a tremendous waste of money.
Your misconception can be explained though: the number of tenured positions is not as much increasing as the number of PhD graduates. Though, this is the same nowadays in industry for leading positions in a company, more and more academic graduates. If you actually like to collaborate, academia is the right place, as it gets more and more interdisciplinary, team-oriented and the number of publications is growing exponentially. Therefore, the direct topical competition has not really become higher, but lower. But it is more a lottery nowadays to become professor. You just have to make a decision if you want to join the lottery game for 5-10 years being a postdoc.
You just seem to have the utter most wrong research strategy: Doing exactly what your peers are doing, just better and faster?! That's exactly the engineering/industry view. Look for unsolved complementary questions in regard to your peers or look/ask for collaborative ideas that make a outstanding contribution to the community and a single group/researcher cannot solve. Also, don't waste public money by doing exactly the same like some other national group. Among different countries there is and has to be competition, due to economical competition.
Most of the funding mondy is also given to the best ideas, not the most competitive researcher, at least if the scrutinization is objective and anonymous, which seems to become more important than having a big name in an interdisciplinary research landscape.
Last but not least, it's not like that the researchers get elected professors which published the most x highest impact factor until 35-40. I know many professors being postdoc nearly a decade before turning professor with 42-45, because they were very well connected in their community and true experts rather than having a couple nature papers with 35. Maybe the latter case becomes more common in times of publish or perish, but this can also be a short trend as many trendy topics in high impact journals when faculties sees that bibliographic statistics are not the best measures to judge the influence of researcher in a community.
Have you ever wondered why many professors in STEM are not 30 year old prodigies, but quite normal and assidiuous people and many chemistry, mechatronics, material science professors being educated physicists? The best and most competitive specialists rather go industry/entrepreneurship and they get paid there much better, interdisciplinary interested and curious researchers tend more towards academia, where the competiton and responsibility is much lower for a professor in comparison to a R&D manager in a company, if you only manage an average research group as a professor and not bigger institutes consisting of several teams and sub-groups.