Exercise from Atiyah - Macdonald about tensor product

The first attempt is fine. The inverse you describe is well defined and the maps are in fact mutually inverse.

Let's have a look at the second approach. (Edited for clarification.)

Tensoring the exact sequence $$0\to \mathfrak{a}\to A\to A/\mathfrak{a}\to 0$$ with $M$ yields the exact sequence $$\mathfrak{a}\otimes_A M\to A\otimes_A M\to A/\mathfrak{a}\otimes_A M\to 0.$$

After identifying $A\otimes_A M$ with $M$ the obvious way, the homomorphisms $\mathfrak{a}\otimes_A M\to M$ and $M\to A/\mathfrak{a}\otimes_A M$ are given as follows. The first, $\mathfrak{a}\otimes_A M\to M$, maps homogenous elements $f\otimes m$ to their product $fm$. It may not be injective, but its image is easily seen to be $\mathfrak{a}M\subset M$. The second map $M\to A/\mathfrak{a}\otimes_A M$ just maps an element $m$ to $1\otimes m$; that the sequence is exact just means that this map is surjective and its kernel is the image of the former map; hence it's kernel is $aM$.

Now, quite generally, if $\varphi\colon N\to N'$ is an $A$-module homomorphism, then the homomorphism $\overline{\varphi}\colon N/\ker{\varphi}\to N'$ mapping a residue class $[n]$ to $\varphi(n)$, is well defined and injective: we have $\varphi(n) = \varphi(m)$ if and only if $n - m\in \ker(\varphi)$. Note that here $\mathrm{im}(\varphi) = \mathrm{im}(\overline{\varphi})$.

Applying this to the map $M\to A/\mathfrak{a}\otimes_A M$, of which we know it is surjective and has kernel $\mathfrak{a}M$, we get the desired isomorphism $M/\mathfrak{a}M\to A/\mathfrak{a}\otimes_A M$. This map, in fact, is nothing but the inverse described in the question.

Arguing via right exactness of taking tensor products is just another way of getting that this map is well defined and bijective. Of course, there are more proofs, mostly because you can make things more sophisticated if you wish, but in their heart, eventually, they will all produce (at least) one of the maps and show that it's an isomorphism.