Definition of a monoid: clarification needed

First definition (algebraic): A monoid is a pair $(M,b)$, where $M$ is a set (called the underlying set of the monoid) and $b\colon M\times M\to M$ is a mapping (called the binary operation of the monoid; for $m_1,m_2\in M$ denote $b(m_1,m_2)=m_1\bullet m_2$), which satisfy the following two properties:

1). for any $m_1,m_2,m_3\in M$ the following equality holds (associativity): $$(m_1\bullet m_2)\bullet m_3=m_1\bullet(m_2\bullet m_3).$$

2). there exists such $e\in M$ (called the identity of the monoid), that for any $m\in M$ the following equalities hold: $e\bullet m=m\bullet e=m$.

It is a standard definition of a monoid. There are a lot of examples of monoids; for example, any group is a monoid. However, there are monoids, which are not groups.

You are right, $(\mathbb{Z},+)$, $(\mathbb{Z},\cdot)$ are monoids, but $(\mathbb{Z},-)$ is not (because, for example, $1-(1-1)\ne(1-1)-1$).

If you are familiar with category theory, then you can get a lot of natural examples in different categories. The reason is following:

Let $A$ be a category, $a\in A$ be its object. Then any morphism $f\colon a\to a$ is called an endomorphism of $a$. Thus we can consider the set of all endomorphisms of the object $a$, denote it by $end(a)$. Note, that we can composite any two morphisms in $end(a)$, therefore we get a binary operation on $end(a)$! $$ \bullet\colon end(a)\times end(a)\to end(a);\qquad f\bullet g=f\circ g. $$ It is a monoid by the definition of category.

Now you can take any category, for example $\mathbf{Set}$, take any its object -- for example, $\mathbb{N}$ (which is considered without its algebraic structure), -- and you get the monoid $\mathbb{N}^{\mathbb{N}}$ of all functions $f\colon\mathbb{N}\to\mathbb{N}$, which, of course, is not a group (check it).

Now it is easy to believe in the following definition of a monoid:

Second definition (category-theoretic): Monoid is a category with one object.

Indeed, denote by $x$ its single object, then $end(x)$ is a corresponding monoid. Conversely, if you have a monoid $(M,b)$, you can define a category $\mathbf{M}$ with single object, such that $Arr(\mathbf{M})=M$, and composition defined by the following rule: $$ \forall f,g\in M:\;f\circ g=f\bullet g. $$

But it was, of course, intuitive reasoning. In order to make an exact statement, I have to give a few more definitions:

1). Let $(M,\bullet_M)$ and $(N,\bullet_N)$ be monoids. Monoid homomorphism is a mapping $f\colon M\to N$, such that for all $m_1,m_2\in M$ the equalities $f(m_1\bullet_M m_2)=f(m_1)\bullet_N f(m_2)$ and $f(e_M)=e_N$ hold. It's easy to check that all small monoids and their homomorphisms form a category, called $\mathbf{Mon}$.

2). Let's denote by $\mathbf{S}$ the full subcategory of $\mathbf{Cat}$, which objects are categories with fixed single object.

Now I can formulate the exact statement (equivalence of two definitions):

Statement: The category $\mathbf{Mon}$ of monoids is isomorphic to the category $\mathbf{S}$.