Is there a connection between the concepts of limits in ordinals, functions and categories?

They are all special cases of limits in the category-theoretic sense.

Limit ordinals are a special case of least upper bounds in partially ordered sets. Given a partially ordered set $(X,\le)$, we may form a category whose objects are elements of $X$ where there is a single morphism from $x$ to $y$ whenever $x\le y$. Transitivity gives us composition and reflexivity gives us identity morphisms. In that case, the least upper bound of some subset $Y\subset X$ is precisely the limit of the diagram spanned by $Y$.

The limits of functions and sequences that we study in functional analysis and, more generally, in topology are in fact also a special case of least upper bounds in partially ordered sets, so they are also generalized by category-theoretic limits. If $X$ is a (topological) space, a filter on $X$ is a set $\mathcal F$ of subsets of $X$ such that:

• $\emptyset\not\in\mathcal F$
• If $Y\in\mathcal F$ and $Y\subset Z$ then $Z\in\mathcal F$
• If $Y,Z\in\mathcal F$ then $Y\cap Z\in\mathcal F$

As an example, if $x\in X$ then the set of all neighbourhoods of $x$ (i.e., subsets of $X$ that contain some open neighbourhood of $x$) is a filter on $X$, called the neighbourhood filter $\mathcal N_x$. We say a filter $\mathcal F$ converges to $x$, and write $\mathcal F\to x$, if $\mathcal N_x\subset\mathcal F$.

What has this to do with convergence of sequences and functions? Well, suppose that $(x_n)$ is a sequence in $X$. Then we can define a filter $\mathcal S_{(x_n)}$ by:

$$ S_{(x_n)} = \left\{Y\subset X\;\colon\;\exists N \;.\;\textrm{if }n\ge N\textrm{ then }x_n\in Y\right\} $$

the set of all subsets of $X$ that eventually contain every term of the sequence. You can check for yourself that $x_n\to x$ if and only if $\mathcal N_x\subset S_{(x_n)}$.

Limits of functions can be handled in a similar way. Now, given some space $X$, we may define a partially ordered set $F$ whose elements are the filters on $X$, ordered by inclusion. Let $\mathcal F$ be a filter whose limit we want to find. For example, we might have $\mathcal F=S_{(x_n)}$ for some sequence $(x_n)$. Given $x\in X$, define $$ \mathcal L_{\mathcal F,x}=\left\{\mathcal G\in F\;\colon\; \mathcal G\subset\mathcal F, \mathcal G\to x\right\} $$ Then $\mathcal F\to x$ if and only if $\mathcal F$ is the least upper bound in $F$ for $\mathcal L_{\mathcal F, x}$.

To connect ordinals and analysis, one puts the order topology on ordinals. Then, given a metric space (or topological space) $X$, a sequence is just a function $f: \mathbb{N}\rightarrow X$, or, using the first countable ordinal $f: \omega\rightarrow X$. It now happens that a sequence converges iff it can be extended to a function $f':\omega +1 \rightarrow X$, such that $f'(n)=f(n)$ for all $n \in \omega$ and $f'$ is continuous with respect to the order topology on $\omega + 1$.

Then, to connect ordinals and category theory, take the category of all ordinals and, say, increasing maps between them. We use the fact that for any two ordinals $\alpha$ and $\beta$, one of them embeds into the other as an initial segment. For any set $A$ of ordinals form a diagram with this embedding maps. Then, the categorical limit of this diagram is precisely the limit in the sence of ordinals: it will be the supremum (limit) of all ordinals from $A$, and the limit maps will embed ordinals from $A$ into this limit.