How did physicists know that only negative charges move?
The Hall Effect shows that negative charge is moving.
In the Hall effect, one passes a current through a wide strip of metal exposed to a perpendicular magnetic field. If positive charges moved, we'd expect the positive charges to be travelling in the same direction as $\vec{I}$, and the magnetic force $q\vec{v}\times\vec{B}$ would be to the right. Thus, we'd expect an accumulation of positive charges on the right of the strip.
However, if negative charges move, we expect they move in the opposite direction of $\vec{I}$, and the magnetic force $q\vec{v}\times\vec{B}$ would again be to the right. This time, we expect an accumulation of negative charges on the right side of the strip.
If you actually do the experiment, you find that the right side of the strip is negatively charged, as shown in the picture. So negative charges are doing the moving!
Image Source: https://www.nde-ed.org/EducationResources/CommunityCollege/MagParticle/Physics/Measuring.htm
Physics's don't know that only negatively-charged particles move. We can create ion currents on demand in many environments. We do know that the current flowing in a metal wire is negatively charged particles in motion.
As for how to determine that, you do a Hall effect measurement.
The measurement works by subjecting a current in a relatively wide bar to a magnetic field perpendicular to both the current and the width of the bar and then measuring the potential difference across the width. In this era of turn-key precision voltage measurements is easy enough to do in a high school laboratory if the students can follow the underlying arguments.
Initially, when first glass rods were systematically being rubbed, the "charging" phenomena was observed. The electric charges were hypothesized to be positive and negative, and the pioneer (Franklin? forgot the name...) pretty much arbitrarily decided to call one positive and the other negative. Further experiments helped him deduce that two like charges repel and opposite charges attract.
At that point, nothing is said about which charge is the moving one. Original assumption was that the positive charges moved and the mathematical formalism reflected this.
Later on came the experiment which deduces which charge is the carrier of current, positive or negative. The experiment is called the Hall Effect. Essentially, you apply a magnetic field directed such that its field is perpendicular to a current flowing in a conductor. Rules of electromagnetism are such that, in this situation the negative charges pile up on one side of the conductor and the positives on the other. By arranging your setup, you can say if left side is negative then that must be the moving charge. (Or vice versa.)
This of course doesn't yet finish the picture. Atomic discoveries established that the protons are stuck in the nucleus, but the hall effect can clearly demonstrate that there are positive moving charge. What gives?
The explanation is that of an imbalance of charges, where positive charge carriers are holes, absence of electrons in a material that shuffle around. It's still the electrons that are moving, but the net motion is that of an electron hole when a material is a positive charge carrying type.