How acceptable is an ellipsis "..." in formal mathematics?

Discussing whether ellipsis are inherently good or bad is not that productive - that's a decision made in reference to particular writing and particular purpose. It is better to recognize that written mathematics is meant to communicate both rigor and intent and to understand the way in which elements like ellipses serve that purpose.

Note that ellipses introduce elements into the text that sums do not:

• They explicitly substitute for the initial (and, if finite, terminal) segments of a sequence, which is useful if you want to make a point about those terms or if those values help clarify that bounds of the sum are sensible.

• They show the ordering of the terms. This is useful if you want to make an argument involving adjacent terms cancelling - and if you're in a non-commutative setting, this is often less ambiguous than a symbol like $\prod$.

• They create some space on the page for each term. This is fantastic when you're dealing with something like generating functions where you might need pointwise operations on the coefficients of multiple series, because you can lay out the coefficients of multiple functions in a grid and can also integrate worked examples of small cases with general calculation by using notation such as $$1+2x+3x^2+\cdots+(n+1)x^n+\cdots.$$

There are also tangential benefits that depend on the audience and purpose - for instance, if you're trying to express a formal argument to an audience without so much mathematical maturity, ellipses can be a nice way to do that. Of course, ellipses also don't do some things that you might want them to:

• Ellipses don't always pin down what the summands are. If the pattern is just "counting, with a function applied unevaluated" - that is an expression like $f(1)+f(2)+f(3)+\cdots+f(n)$ - it's probably safe, but one has to be careful not to frustrate readers. Of course, you can always include a general term to clarify or explicitly state your intention in the text preceding the equation (and, hopefully, the equation should not come from nowhere! If it does, you haven't written enough words to introduce it!)

• Ellipses do not indicate the indices of the sum. This can be relevant if you need to split up the sum in some way, as often happens in analysis - there's no good way to say "here's the set of big terms, and here's the set of small terms, let's look at them separately."

• Ellipses cannot represent sums without order. If you're summing over the set of partitions of some set, you'd better use summation notation.

There are surely more subtle things, but these are the most striking aspects of the notation that come to mind that would most often persuade me to use ellipses or to avoid them - and there are definitely situations where a creative use of notation can contradict what I wrote here and situations where it doesn't really matter what notation you choose.

I have two points. Firstly, this question is inherently subjective. Different people have different opinions. You should be aware of this, and form your own opinions about what you do and do not care about. You should also be aware that your opinions may annoy other people no matter what you do. For example, if you use an ellipsis you may annoy someone but another person may look favourable on your choice, while not using one may produce the polar opposite effect.

Secondly, I do not believe that research papers are the correct place for "formal mathematics". A paper is a means of communication, and therefore should be readable. I believe that it is more helpful that the reader should be able to reconstruct the formal mathematics from the arguments presented. The skill in writing good mathematics is to write something readable, and where this "reconstruction" is easy. Therefore, ellipses have a place in research papers.

(Note that the above paragraph has lots of caveats missing. Hopefully you can reconstruct my intended point from it though...)

The following ten aspects are relevant concerning the question:

1. If there is (almost) no ambiguity for an informed reader and if it increases the readability, there is no problem with using an ellipsis.

...

10. If you can replace an ellipsis by a more rigorous notation without increasing the complexity / decreasing the readability, then you should do so.