Find the remainder of $2^{11}$ by $23$

$\left( \frac{2}{23}\right) = 1 \Rightarrow 2^{11} \equiv 1 \left( \bmod 23 \right)$

Hint $\rm\ \bmod\ 23\!:\ \ 2 \equiv 5^{\large2}\, \Rightarrow\, 2^{\large11} \equiv 5^{\large 22} \equiv 1\ $ by Fermat's little Theorem.

See Euler's Criterion and quadratic reciprocity to understand what happens generally.

Regarding square-roots, $\rm\ x^2 = a^2\ \iff\ (x-a)\ (x+a) = 0\ \iff\ x = \pm\: a\ \ $ holds true in any integral domain, i.e. it's true in any ring without zero-divisors. More concretely, in $\rm\ \mathbb Z/p\:,\: $ we have prime $\rm\ p\ |\ (x-a)\ (x+a)\ \Rightarrow\ p\ |\ x-a\ $ or $\rm\ p\ |\ x+a\:,\: $ so $\rm\ x \equiv \pm\: a\ \ (mod\ p)\:.$