A basic question about $\Rightarrow$

Lets do an example with something other than $\textrm{PA}$.

Say $\textrm{GT}$ is group theory. Say $A$ is "$\forall x, x^2 = e$", that is, every element has order $1$ or $2$. Say $B$ is "$\forall x, x^3 = e$", that is, every element has order $1$ or $3$.

Now $\textrm{GT} \vdash A$ and $\textrm{GT} \vdash B$ are both false. (Group $\mathbb Z/5$ fails both properties, so of course $\textrm{GT}$ cannot prove either one.) Thus

$$ \textrm{GT}\vdash A\;\Longrightarrow\; \textrm{GT}\vdash B\quad\quad(*) $$ is true. On the other hand, GT cannot prove $A \rightarrow B$. (Group $\mathbb Z/2$ satisfies $A$ but not $B$). Thus $$ \textrm{GT}\vdash A\rightarrow B\quad\quad(**) $$ is false.

---

Advice: Don't try to understand logic applied to arithmetic, until you already understand logic applied to more mundane topics (like group theory).