Why do we want complete spaces? We don't we just use closed spaces?
First, a peripheral technical issue: Unlike completeness, closedness is not an absolute property; it is a relative property. A space $S$ is complete or not complete. But it makes no sense to say that $S$ is closed or not closed. All you can say is that it is closed relative to some larger space $T$. For example, consider the set $(0, \infty)$ of positive real numbers. This is not closed relative to $\Bbb R$, because it fails to contain its limit point $0$. But it is closed relative to $\Bbb R\setminus\{0\}$ because now it does contain all its limit points—zero is no longer a limit point because we removed it from the space. And of course it is closed relative to $(0,\infty)$ because every topological space is a closed subset of itself. So your suggestion does not even make sense, because there is no such thing as a "closed space".
(The issue can be seen more generally if you consider the definition of a closed set: $C$ is closed if $X\setminus C$ is open. But this depends on what $X$ is.)
But there is a larger issue. The Greeks noticed 2300 years ago that the rational numbers are incomplete. Putting it anachronistically, they noticed that there are Cauchy sequences that do not converge. A theory about such spaces is needed. You asked
what is the benefit in introducing this weaker notion of complete spaces and dealing with these weaker types of sequences called Cauchy sequences
This is like asking why physicists spend so much time trying to understand friction forces, when everyone knows the problems are much easier to solve if you simply assume that there is no friction. Why don't physicists take the easy way out? Because there is friction and the whole point of physics is to solve problems about how the world really is, friction and all.
Like physics, mathematics is not just made up from nothing. We want to solve certain kinds of problems and understand the way numbers and shapes work. The rational numbers seem to be a fundamental sort of object, one of the things that is important in the structure of the universe. The whole point of mathematics is to understand how things like the rational numbers work and how they are related to other things like the real numbers.
And the most important part of the answer to that question turns out to be: the rational numbers contain sequences where the elements get closer and closer together (cauchy sequences) and sort of converge, even though there is no rational number that they converge to. (A common example is $1, \frac32, \frac75, \frac{17}{12}, \frac{41}{29}\ldots$, with $\frac ab$ followed in each case by $\frac{a+2b}{a+b}$. We can show that if this sequence were to converge to some limit $L$, we would have $L^2 = 2$. But we have known for a long time that there is no such $L$.)
But the real numbers behave differently: if elements of a sequence get closer and closer, there must be some single point $L$ that they get close to. The two kinds of numbers are fundamentally different in this way, and we are stuck with it, just like we are stuck with friction. We need to give that difference a name. The name is “completeness”.
Note that completeness is a metric property whereas closedness is a topological property, and hence, depends on the topological space with which it is referred to. So, any set can be made closed by choosing the appropriate topology and as noted in the comments, every set is closed in itself. But, not all sets can be made complete by changing the metric maintaining the topology, an example being $\mathbb{Q}$, the set of rational numbers. Thus, completeness is crucial in discussion involving convergence, which is what mathematical analysis is most associated with. As pointed out in the comments, completeness is an intrisic property, and points must be added to make a non-topologically complete/non completely metrizable space complete. The historical importance of completeness comes noticing that all major branches of Analysis-Real, Fourier, Functional etc. have developed by completion of spaces that were hitherto not complete: The concept of irrational numbers was the first notion of completeness, though introduced without the exact notion of completeness, but nevertheless the idea was completion. The theory of Lebesgue integral is nothing but born out of completion of the space of Riemann integrable functions, The convergence theorem of Fourier series requires the completeness of $L^2(\mu)$ for its proof and the list goes on.
If the space is closed we know that the limits of a sequence exist
That's... false. Try $0,1,0,1,\ldots$.
Did you mean that every sequence has a convergent subsequence? I hate to break this to you, but that's only true if the closed set is a subset of a complete space.
Or did you mean that every Cauchy sequence has a limit? Again... that's only true in a complete space.
Completeness is important, you just have only been using examples of closed sets that are in complete spaces. And it sounds like the properties you "like" are actually properties of complete spaces. Bluntly you are confusing a lot of related concepts.
- "Closed" is a topological property. "Complete" is a property of metric spaces only. $[0,1]$ is closed in $\mathbb R$ and $[0,1] \cap \mathbb Q$ is closed in $[0,1] \cap \mathbb Q$.
- "Closed" only makes sense relative to a containing topological space. "Complete" is an intrinsic property.
- "Limit points" can be defined just in terms of open sets and topology. Closed sets contain all their limit points in any topological set.
- "Completeness" means if a sequence tries to converge, then it hs something to actually converge to. "Convergence" only makes sense in metric spaces. This is not true for $\mathbb Q$ where sequences might try to converge to $\sqrt 2$ for instance. (1, 1.4, 1.41, 1.414... is such a sequence).