How do you minimize stepper motor vibration?
What you need is sinusoidal current drive.
In other words, you have to treat the motor like a traditional brushless motor, rather then a stepper. This requires pretty specialized stepper drivers, and is not simple.
A simpler alternative might be to try microstepping the stepper motor, but that won't get you perfectly smooth rotation.
Really, for situations where you need extremely smooth rotary motion, a stepper is really just the wrong control system. You should use a brushless AC motor, or at least a brushed DC servomotor.
Here is a decent white-paper on stepper drive modalities, with some contrasting to AC synchronous motors.
Using full step driving, the rotor acts much like a Spring-Mass-System, with the rotor being the mass and the magnetic force being the spring. When you move from one step to the next, the motion will always be rough. The rotor pretty much jumps from one step to the next and it takes some time until the spring dampens out the rotor's energy, causing a little oscillation (read: rough motion).
You can smooth this when you use half-step mode, and you can additionally compensate the torque-nonlinearity, cf. this link
Following this logic, you eventually end up using fine-Stepping, micro-stepping and sinusoidal driving. (See this link for micro-stepping)
Some more details:
The resonant frequency of a stepper motor's rotor is usually somewhere around 50 Hz ... 400 Hz. When you drive the motor in full-step mode at its own mechanical resonant frequency, things will get pretty bad and it is likely that you lose (jump over) steps. For slow speeds, it is a good idea to stay below the motor's resonant frequency. For high speeds, try to get beyond the resoncance as fast as you can while accelerating, and don't use full-step driving.
I can only suggest reductors, if speed is not a limiting factor.
Otherwise I would go for brush-less motors with some feedback.