M compact $p\in M$ , there exist $f:M-p\to M-p$ continuous bijection but not homeomorphism?
I will denote by $C(\omega)=\{0,1,2,\dots\}\cup\{\omega\}$ the one-point compactification of the discrete space on the countable set $\{0,1,2,\dots\}$.
I.e. all points different from $\omega$ are isolated and neighborhoods of $\omega$ are precisely complements of finite subsets of $\{0,1,2,\dots\}$.
(This space is homeomorphic to the space $\{0\}\cup\{\frac1{n+1}; n=0,1,2,\dots\}$ with the topology inherited from real line. So this is simply a convergent sequence.)
Now I take $M=\{0,1\}\times C(\omega)$, where $\{0,1\}$ has the discrete topology. (I.e., $M$ is the topological sum of two copies of $C(\omega)$.) And I choose $p=(0,\omega)$.
It is easy to see that $M$ is a compact metric space.
(E.g. it is homeomorphic to $\{0,1\}\times(\{0\}\cup\{\frac1{n+1}; n=0,1,2,\dots\})$ with the topology induced by the usual Euclidean metric on $\mathbb R^2$.)
Note that every subset of $\{0\}\times C(\omega)$ is open in $M\setminus\{p\}$.
Let us define a map $f \colon M\setminus\{p\} \to M\setminus\{p\}$ by putting $$f(0,2n)=(1,2n)\\ f(0,2n+1)=(0,n)\\ f(1,n)=(1,2n+1)\\ f(1,\omega)=(1,\omega).$$
This map is bijective and continuous, but the extension $\overline f\colon M\to M$ with $\overline f(0,\omega)=(0,\omega)$ is not continuous, since the sequence $x_n=(0,2n)$ converges to $(0,\omega)$ in $M$ but $\overline f(x_n)=(1,2n)$ converges to $(1,\omega)\ne \overline f(0,\omega)$.
Here's my attempt to sketch the above map (the two big arrows indicate the direction in which the sequences $C(\omega)\times\{0\}$ and $C(\omega)\times\{1\}$ converge):
It is perhaps worth mentioning that $f^{-1}$ is not continuous. The sequence $(1,2n)$ converges to $(1,\omega)$ but $f^{-1}(1,2n)=(0,2n)$ does not converge.
So this does not contradict Theorem 29.1 from Munkres, which was mentioned in comments.
Theorem 29.1. Let $X$ be a space. Then $X$ is locally compact Hausdorff if and only if there exists a space $Y$ satisfying the following conditions:
- $X$ is a subspace of $Y$.
- The set $Y-X$ consists of a single point.
- $Y$ is a compact Hausdorff space.
If $Y$ and $Y'$ are two spaces satisfying these conditions, then there is a homeomorphism of $Y$ with $Y'$ that equals the identity map on $x$.
The OP indicated that he was originally thinking about the case that $M=S^n$. Since $S^n$ is the one-point compactification of $\mathbb R^n$, from Theorem 29.1 and from the fact that every continuous bijection $\mathbb R^n\to\mathbb R^n$ is a homeomorphism (which was shown in this question) we get that in the case $M=S^n$ the result from the question is true.