Why is this derivation of Schmitt trigger circuit's output signal, incorrect?

You made an assumption there's a stable output and calculated what the output should be in case the assumption is right. In addition you clipped the result to the possible output voltage range. The clipping is ok, but the assumption of the existence of stable output is not, as persons who know the feedback stability theory could confirm (see NOTE1)

People make same kind of reasoning continuously. Actually the whole physics is based on this kind of reasoning. There comparing to measurements is the way to reveal wrong assumptions.

NOTE1: there's no need to be an academic level mathematician or engineer to be able to make it clear a stable output can be achieved only because the limited voltage range clips the output. Some elementary Laplace domain analysis is enough.

If we assume there's some realistic slowness in the amp, say one RC charging and the gain is finite, maybe big, but finite we can find the transfer function for the whole circuit. Th slowness prevents infinitely fast changes so we can follow what the circuit does.

We can replace the ideal amplification A with G/(1+sRC) which is the transfer function of buffered RC integrator. G is the DC gain of the amp.

Let's also simplify the formula by replacing R1/(R1+R2) with a single symbol B. It's our feedback attenuation factor which is between 0 and 1.

The s-domain gain of the system is Vo/Vi = 1/(B-(1+sRC)/G)

Of course the output stays zero if the input is zero and there's no noise. But there's always some noise. We can find which s-domain frequencies start to ring in the system from the slightest pulse of noise by calculating which values of s make the denominator infinite (=find the poles of the transfer function). We solve s from equation (B-(1+sRC)/G)=0

The result is s=(GB-1)/RC

The Laplace transformation math says that the output from a slightest pulse of noise is proportional to an exponential voltage exp(t/T) with time constant T=RC/(GB-1). This T is positive as soon as GB is bigger than 1. Positive time constant means limitless grow which is in practice stopped only by the limited output voltage range. Negative T (i.e. GB <1) means that the ringing in the loop decays and the output stabilizes to the value which can be calculated with your original formula for Vo. But for a stable output A must be less than how much the feedback voltage divider attenuates.

Point 1

The Schmitt trigger has hysteresis. Hysteresis implies that the circuit has memory. It remembers it last state. For a system with memory, you cannot write \$V_o = f(V_{in})\$. It should be of the format \$V_o = f(V_{in}, V_{o, \text{prev}})\$ or something equivalent._{As one of the comments mentioned below the question indicates, one would not know the system has memory the first time they try to solve the circuit using equations. IMHO, In that case, the following section would safeguard against an erroneous conclusion.}

Point 2

Output voltage being able to saturate is also an important feature since it prevents \$V_o\$ and \$V_x\$ reinforcing each other to infinity. Your equations do not model the saturation non-linearity.

Your second equation would have been better written as

\$ V_o = \min(\max(A(f(V_o) - V_{in}), -V_{max}), V_{max}) \$

With this scaffolding to represent the non-linearity, all further simplification attempted in the question would have been prevented.

edit

_{In response to OP's question below in comments.}

Let's analyse the case where \$V_{in} = 0\$. OP's second equation simplifies to

\$V_o = A(\frac{R_1}{R_1+R_2}V_o - 0)\$.

Neglecting the saturation, and for \$A\frac{R_1}{R_1+R_2} > 1\$, the solution to this system is

\$V_o = 0\$ or \$V_o = \infty\$ (since \$0 = A\frac{R_1}{R_1+R_2} \cdot 0\$ and \$\infty = A\frac{R_1}{R_1+R_2} \cdot \infty\$).

This means that, if the opamp output is forced to 0 and if there is no noise (or any other imperfection) in the system, the output remains there (OP's waveform also show zero volt output for zero volt input).

In a practical circuit, the output will be displaced from 0 volts by noise. So the question is, will the system remain there? Will the system move back to zero volt or \$\infty\$ volts? Dynamics (time evolution) of the system is not modelled by OP's equations, so, we cannot answer this question keeping ourselves to the algebraic equations where time is not modelled. If time was also modelled, I think we could have concluded that the 0 volt equilibrium point is unstable and the \$\infty\$ volt equilibrium (or \$V_{max}\$) is stable and system will have tendency to move towards the extreme output situation.

In short, using the algebraic equation above, we cannot analyse the this circuit when the output is not touching the saturation values (\$-V_{max} < V_o < V_{max}\$) since a practical system will have tendency to shift towards the saturation points and not lie exactly on the solution to the algebraic equation above.

edit 2

_{In response to comments below which ask to forget hysteresis stuff. I am attempting to construct an example without hysteresis}

Let me try to make a point with an analogy where an algebraic solution exists, but output is unbounded. This analogous system also has positive feedback. It too has finite output predicted by the equation. But output is unbounded.

The output-input relation is given by

\$ \begin{align} \frac{dy(t)}{dt} ={}& x(t) \color{red}{+} y(t)\\ (s-1)Y(s) ={}& X(s)\\ \frac{Y(s)}{X(s)} ={}& \frac{1}{s-1} \end{align} \$

For any finite amplitude sinusoid signal (including 0 frequency), the output predicted by the transfer function is finite. But the system will have unbounded output. The gain of this system as a function of frequency is same as the system \$\frac{1}{s+1}\$. I think this example forms a nice parallel to your example. No hysteresis or saturation was used in this example.