Is light amplitude spatial?

If the person drawing the graph bothers to label the axes you'll see that the thing that "goes up and down" is not displacement as it is in a wave on a string but electric field strength.

So, no, nothing is moving off the line of the ray, but the because electric field is a vector the oscillation does have a direction associated with it (and therefore polarization makes sense).

There is a pretty picture in wikipedia

Electromagnetic waves can be imagined as a self-propagating transverse oscillating wave of electric and magnetic fields. This 3D animation shows a plane linearly polarized wave propagating from left to right. Note that the electric and magnetic fields in such a wave are in-phase with each other, reaching minima and maxima together

On the axis is the direction of the plane wave. What is increasing and decreasing in space are the electric and magnetic fields. As the energy that the wave carries is proportional to the average electric field squared, what is increasing and decreasing in this polarized wave is related to the energy.

In non polarized light the rays have random polarizations so the wave is not organized macroscopically but a lot of directions can be visualized, in the plane perpendicular to the direction. The energy carried is still proportional to the average electric field squared, but cannot be drawn in a pretty diagram.

Here's an animated flux diagram, a cross section of the e-field pattern radiated by a short dipole emitter, from the MIT physics 8.02 course.

Here's a 3D flux pattern frozen in time. (Also lots more cool stuff from MIT 8.02)

Notice, no sine waves. Just expanding blobs (they'd be donut-shaped in 3D,) with the maximum wave-emission being broadside, and a zero node on the vertical axis. Near the EM source, the field pattern looks like expanding tori with the e-field wrapped around poloidially. Farther away from the dipole source we'd see it more as thin, expanding sphere waves with holes at the poles, with e-field and b-field flux lines "drawn upon the sphere," and at right angles to each other. The flux lines always close upon themselves to form squashed, outward-moving circles.

I suspect that these hand-waving sine-wave explanations (and even textbook explanations about "transverse waves") date to many decades ago, back when all light was "Transverse Waves In The Aether." Light is not a transverse wave, not like a shaking rope or shear-wave acoustic vibrations in solids. But it's very hard to make physics textbook authors change their language (search SJ Gould and the Fox Terrier clone problem.) The luminiferous aether was struck down, yet few authors stopped using the vibrating-string analogy for polarization, or stopped teaching us that EM radiation is "transverse wave."