In a mathematics publication, is it preferable to state all definitions immediately, or as they arise?

There is no general rule. Different authors have different opinions about this. To my knowledge no mathematics journal has guidelines about this sort of thing.

A rule of thumb could be that if your notation is going to be used throughout the paper, then it should be introduced at the beginning in a special section called "notations", "conventions", "background", or however you want to say this. If the notation is only going to be used locally in some section or even one proof, then I would introduce it at that point.

Another possibility is to make a categorized table of notations at the beginning or an alphabetical list of notation with references to where the object is defined, if your paper is long and complex enough to justify it. This can of course be combined with the previous rule of thumb.

I would recommend against making the "background" section as part of the introduction. The introduction is supposed to be something that can be read by almost everyone slightly interested in the paper, to know what the main results are and how they fit with the general literature, and in case of stars aligning, whether the paper is worth reading. (The abstract is to decide whether you'll even glance at the paper at all.) Keep it short and simple and avoid introducing too many things at once. In my opinion it is a mistake made by many younger, and also more experienced authors to make the introduction way too technical.

What you should prioritize in any paper, mathematics or otherwise, is readability. Organize the paper so that your reader can comfortably follow your argument without a lot of jumping around.

In a short, relatively "flat" paper, almost any organization will probably be ok. Flat in the sense of later parts not depending fundamentally on earlier parts.

But in a longer paper it becomes an issue, for me at least. Things should be introduced with some context: Why is this being introduced here. If I see a definition, I expect to be able to easily grok why it is here from some prior context or by being used quite soon in the development. A definition or piece of notation introduced without context is just irritating.

As an extreme example, imagine a calculus or abstract algebra textbook in which all of the book's definitions are in the first chapter along with every notation that will be used.

I'll note that the very purpose of defining things in mathematics is to give us something to think (and write) about. Defining something with no context is just noise. We define rational number or Abelian group for example, because we want to say things about them. If you define them, but don't soon discuss them, the reader has no context.

So, first think about the flow of the paper from the standpoint of the reader. I think that in most work with any significance the "just in time" organization will probably work better. I'll admit there may be exceptions. But it is the readability that should be prioritized, not some abstract concept of an "ideal" organization.

Everything you write in a paper should be the answer to the question that is in the reader's mind at that point. So the title needs to answer the question: "Do I bother with this at all?". The abstract, as @user119516 says, answers "do I glance at this?", and the introduction answers "do I read this?".

If you follow that principle then it is obvious that you do not divorce your definitions or novel notation from the points in the paper where they are used, because until the reader reaches those points they have no question in their mind about them.

If your paper is very long then it might help the reader if you collect all your notation and maybe your definitions into some helpful appendix, but don't interrupt the flow of the paper with such stuff.

Fashion plays a part here. At one time, and maybe still in some sub-disciplines, it was considered reasonable for the first sentence of a pure mathematics paper to be something like : "Let A be set ...". No-one writing with the reader in mind would do so, surely.