Why only sine waves?
Because sinusoids have some important mathemtical properties. The first being how they behave under differentiation and integration.
$$\frac{d}{dt}\sin(\omega t+\varphi) = \omega\cos(\omega t+\varphi) = \omega\sin(\omega t+\varphi+\frac{\pi}{2})$$
In other words when we differentiate or integrate a sinusoid we get a sinusoid of the same frequency. The sinusoids are the only periodic functions (from the reals to the reals)* for which this is true.
The second being how they behave under addition. Two sinusoids of the same frequency but different phase add together to make a sinusoid of the same frequency (unless they are equal and opposite in which case they cancel to produce zero).
$$a\sin(\omega t)+b\sin(\omega t+\theta)= \sqrt{a^2 + b^2 + 2ab\cos \theta} \sin(\omega t+\operatorname{atan2} \left( b\,\sin\theta, a + b\cos\theta \right))$$
These properties mean that when we feed a sinusoid into a linear time invariant system we get a sinusoid of the same frequency out. Many real-world systems behave to a first approximation as linear time invariant systems, especially for small signals. We can characterise a linear time invariant system by measuring its magnitude and phase response to a sinusoidal sweep and then we can predict its response to other signals by breaking those signals down into combinations of sine waves and then applying the superposition principle.
If we tried to do a similar frequency sweep test with any other waveform we would have an output waveform a different shape to our input waveform, which we would have to deal with somehow, making the characterisation process much trickier.
* As has been pointed out in the comments the exponential is it's own derivative, but the exponential of a real variable is not periodic. The exponential of a real variable multiplied by the imaginary unit is periodic but produces a complex result. If we decompose it into it's real and imaginary parts using Euler's formula then we are back to a pair of sinusoids.
If we apply a sinusoidal signal into a linear time-invariant system (LTI), the output of that system will also be sinusoidal, of same frequency, but possibly different phase and magnitude. If we apply an input that can be described as a sum of sinusoids, output will also be the sum of sinusoids of same frequency, possibly different phase and magnitude. This makes it very easy to characterize the system in terms of phase and magnitude responses.
Using Fourier series, we can build any periodic waveforms with sinusoidal signals. This adds to the attractiveness of using sine as a test signal. We get to know the response of any periodic waveform if we know the response to a sinusoidal signal.
As to the second question, other signals like step and ramp signals are also used as test inputs. However, these signals does not enjoy the privilege of sine as these are not eigen values of LTI system. The application of a test signal depends on what we are trying to see. For example, a step signal is applied to see how the output reacts to a sudden change in input.
A pure sine wave is an useful test signal because it has a special property, it contains only energy at a single frequency, while other waveforms contain energy on multiple frequencies. So depending on what is being tested, a sine wave or other waveforms may be used.
With a sine wave generator and a tool that can simply measure amplitude of sine wave (e.g. multimeter, oscilloscope), you can measure ratio of output and input amplitudes with sine waves of different frequencies to find out frequency response or bandwidth of a system under test.