Photon absorption by atoms vs compton scattering

A free electron does not have any potential to which it is constraint so it can have any energy possible. i.e. it's energy states are continuum much like a free particle in classical mechanics. So scattering of such a particle is possible.

On the other hand, when in an atom, the electron is subjected to coulomb potential and this potential dictates the energy levels accessible to the electron. It's no more continuous. This means that the electron can only absorb or emit energies in steps. Thus it's an all-or-nothing phenomena. What are the energy states accessible is dictated by solving Schrodinger's equation. Thus any photon cannot pluck out the electron only a specific protons can do that(for whom the energy matches the energy difference of states).

For the second part of your question, In Compton effect the change of wavelength is determined by the scattering angle. The more the angle the more is the change. And you can treat the initial wavelength as the free parameter in this case.

In Compton scattering the energy of the incident photon is very much greater than the "binding energy" of the target electron so the target electron can be considered as an essentially stationary "free" electron and its final state is not being bound to the nucleus that it was originally bound to. So there is no well defined energy level that it must jump to.
In the "all or nothing" bit of you question I assume you are writing about the promotion of an electron between two well defined energy levels which requires a photon of a well defined energy.

In the Compton experiment the detector is placed at a particular angle and the intensity of the radiation detected is plotted against the wavelength of the radiation detected.
Two peaks are found. One of them which occurs at a wavelength very close to the incident photon wavelength as a result of the incident photons colliding with tightly bound inner shell electrons and the other due to Compton scattering.

The wavelength of the scattered photon is determined by the laws of conservation of energy and momentum.

More here