What type of encoder is this (optical, magnetic, inductive... etc)? and how does it operate?

You'll notice that there are 8 magnets, but the "coil" (the serpentine traces) have 3-fold symmetry. This means that as the magnets rotate, the induced voltage in the pickup will vary with rotation, with a period 3 x 8 or 24 (EDIT - oops, 12, since the magnets need to alternate north and south) times the shaft rotation rate. This varying voltage is AC-coupled to the first op amp, where it is amplified and passed to the second. The second adds more gain and effectively half-wave rectifies the signal since the op amp is running single-ended. The output of the second op amp drives the base of the output transistor, so you get 24 (EDIT - 12) open-collector pulses per revolution.

Depending on the gains involved and the strength of the magnets, this will only provide reliable operation over a particular range of shaft velocities, so it is technically a shaft encoder, but unlike optical encoders it can only be used for velocity, not position, control.

It is probably used to provide precision locking of the motor to a reference speed, which cannot be done with "standard" tachometers due to voltage coefficient uncertainties and line loss, or to provide a precision rotation signal and is possibly intended for an application like use of a chopper wheel with a lock-in amplifier.