How to solve $a$ in $\int_a^xf\left(t\right)dt=x^2-2x-3$

Method 2(MUST SEE!)

Substitute $x=a$ onto both sides of your original equation.

$\int_a^xf\left(t\right)dt=x^2-2x-3$ becomes $\int_a^af\left(t\right)dt=a^2-2a-3$.

From this, you can IMMEDIATELY notice that $0=a^2-2a-3$, providing you the solutions.

Method 1 (your approach)

Differentiating both sides with respect to $x$, $f(x) = 2x-2$.

And thus, $\int_a^xf\left(t\right)dt=\int_a^x\left(2t-2\right)dt=x^2-2x-(a^2-2a)$

Comparing this to the original equation, $x^2-2x-3=x^2-2x-(a^2-2a)$

You can solve it from here.

Well after differentiating you know: $$f(x) = 2x-2$$ and therefore $\int_a^x 2t-2 dt = t^2-2t|_a^x = x^2-2x-a^2+2a$.

From this immediately follows: $a^2-2a = 3$ with the solutions $a=-1$ and $a=3$