Is there a model exists, such that homomorphic to the set of real numbers but the order is given by $\in$?

The axiom of foundation tells us that $\in$ is a well-founded relation, i.e., there are no infinite descending $\in$-chains in the universe of sets. Since $\mathbb{R}$ (and any model of the theory of $\mathbb{R}$) contains infinite descending $<$-chains, this gives a negative answer to your question. More precisely: if a set $M$ is linearly ordered by $\in$, then $(M,\in)$ is a well-order. In particular, it is order isomorphic to some ordinal.

On the other hand, if we drop the axiom of foundation from ZFC, it is consistent that there is a set $M$ which is linearly ordered by $\in$ and order isomorphic to $(\mathbb{R},<)$. Aczel's anti-foundation axiom (which is relatively consistent with ZFC without foundation) asserts roughly that the membership relation on the transitive closure of some set can look like any directed graph. We can apply this to the directed graph $(\mathbb{R},<)$, thinking of the instances of the order relation $<$ as directed edges between real numbers.

The membership relation is well founded, which lends itself particularly well to being an ordering relation on the naturals; but the ordering relation on the reals is definitely not well founded. So your hunch is correct, this can't be done.