Definition of neighborhood and open set in topology
One of the problems of introducing students to topology is that the open set axioms are often taken as THE definition of a topology, when they are quite unintuitive, though extremely useful in the long run. I argue that the neighbourhood definition, while somewhat cumbersome, has the advantage of being closely related to ideas from analysis, and has a historical basis; it is of course as follows:
A neighbourhood topology on a set $X$ assigns to each element $x \in X$ a non empty set $\mathcal N(x)$ of subsets of $X $, called neighbourhoods of $x$, with the properties:
If $N$ is a neighbourhood of $x$ then $x \in N$.
If M is a neighbourhood of $x$ and $M \subseteq N \subseteq X$, then $N$ is a neighbourhood of $x$.
The intersection of two neighbourhoods of $x$ is a neigbourhood of $x$.
If $N$ is a neighbourhood of $x$, then $N$ contains a neighbourhood $M$ of $x$ such that $N$ is a neighbourhood of each point of $M$.
Then one says a function $f: X \to Y$ is continuous wrt neighbourhoods on $X$ and $Y$ if for each $x \in X$ and neighbourhood $N$ of $f(x)$ there is a neighbourhood $M$ of $x$ such that $f(M) \subseteq N$. The open set definition of continuity is then justified as being equivalent to this definition in terms of neighbourhoods.
One also says a set $U$ in $X$ is open if $U$ is a neighbourhood of all of its points.
THEN one can develop the open set axioms and show that one can recover the neighbourhoods.
Students should be aware that there are many approaches to the notion of topology, whose advantages should be compared. There should be no "take it or leave it" approach, but students should be encouraged to form a judgement, in terms of the character of the theory and its methods. And see which definition is appropriate in which cases.
June 14: The above approach is taken in my book Topology and Groupoids, in order to motivate the definition of open set.
November 17, 2016. Peter Freyd writes in the Introduction to his book Abelian Categories
"If topology were publicly defined as the study of families of sets closed under finite intersection and infinite unions a serious disservice would be perpetrated on embryonic students of topology. The mathematical correctness of such a definition reveals nothing about topology except that its basic axioms can be made quite simple. And with category theory we are confronted with the same pedagogical problem. ......
A better (albeit not perfect) description of topology is that it is the study of continuous maps; and category theory is likewise better described as the theory of functors. Both descriptions are logically inadmissible as initial definitions, but they more accurately reflect both the present and the historical motivations of the subjects."
I would also like to put in a reference to remarks of Bill Lawvere that the notion of space in mathematics is crucial for the representation of motion. This is illustrated in this lecture Out of line, particularly the section and video on Motion.
26 January, 2020
I would like to add another reference, to "Indiscrete thoughts" by G-C Rota. He contrasts a definition with a description (p.48). The book has many other important points to make!
2 May, 2020 I'll also mention that a topology can also be axiomatized in terms of the closure operation, as well as in terms of Int and of Ext. Students should be encouraged to think and evaluate, and not just accept an authoritarian viewpoint. (there is a volume of Progress in Commutative Algebra 2, Closures, Finiteness and Factorization, with four editors, published by de Gruyter in 2012, 328 pages, and available online; the Preface has some good remarks on analogies in mathematics).
13 May, 2020 I should also add the notion of filter, and that link for more detail, to the ways of axiomatising topological spaces, and more.
One defines a topology on a set by specifying the open sets.
Let $X$ be a set. If $\tau$ is a family of sets with the following properties, it is called a topology.
- $X$ and $\varnothing$ are in $\tau$
- Any (possibly infinite, even uncountably infinite) union of sets in $\tau$ is in $\tau$.
- The intersection of any finite number of elements of $\tau$ is in $\tau$.
We call the sets in $\tau$ the open sets. You can see that the collection of open sets in, for example, $\mathbb{R}^2$ has exactly this set of properties.
A neighborhood of a set $S$ is a set $P$ that contains an open set $U$ so $S\subset U\subset P$.
For more information, see Topology by Munkres.
To complement the other answers, which tell you what the normal definition of open set in a topology, I'll give another possibility for the definition of neighbourhood in a metric space (note that this won't make sense for general topological spaces, but I think it's what's motivating the definition of open set you gave).
For a point $p$ in a metric space $(X,d)$, say that a subset $U\subset X$ is a neighbourhood of $p$ if there exists $\varepsilon>0$ such that $B(p,\varepsilon)=\{x\in X:d(x,p)<\varepsilon\}$ is a subset of $U$. Now the definition of open set you've given agrees with the usual one for metric spaces.