Is thermodynamics only applicable to systems in equilibrium?

It entirely depends on what you think "thermodynamics" is.

The traditional idea of thermodynamics dealing with systems whose macrostate can be fully described by e.g. temperature, pressure and volume indeed only applies to systems in equilibrium. Of course, as an approximation it also applies to systems "not far" from equilibrium, for some suitable notion of "not far", explaining its success in describing nevertheless a plethora of phenomena that occur in the real world.

However, non-equilibrium thermodynamics also exists, and is well and alive as a subfield of both classical and quantum physics. Its methods, however, differ strongly from what is commonly referred to as "thermodynamics" in introductory textbooks.

Strictly speaking thermodynamics only describes systems at equilibrium or systems which undergo some change, at the end of which they have time to relax back to an equilibrium state. The signature of a thermodynamic system is the huge reduction of the number of degrees of freedom required to describe the state of the system.

Non equilibrium is a weak characterization and actually one can distinguish different levels of departure from thermodynamic equilibrium.

For example, hydrodynamics corresponds to a case where, due to a macroscopic movement of the fluid, if the flow is not too complex, a three dimensional velocity vector field is required, in addition to a couple of thermodynamic scalar fields which describe the local thermodynamic equilibrium.

The special case of thermodynamic systems brought slightly out of equilibrium is also interesting. In that case it is possible to study the fluxes which try to restore equilibrium and to get information about transport coefficients.

However, one has to take into account that major departures from equilibrium are possible, basically requiring to go to detailed descriptions using a huge number of degrees of freedom.

Good question. The study of thermodynamics is usually between eqilibria. But these take many shapes. And they are yours to define - a state of chemical equilibrium is for instance a state where we define that a forward reaction is equal to its reverse reaction. But we can define only forward flux as a state - an adiabatic state is where a chemical reaction is isolated from heat and work from the outside - so that the only place for heat generated by the reaction is the heatflux of it's reactants. This is certainly not a chemical equilibrium per se but it is a defined state. What is very useful for aspiring thermodynamicists is that we can daisy-chain states together even if we do not know the relationship between state A and state C, as long as we know something about the relationships A and B in addition to B and C, we can go that way.

So, most of thermodynamics are basically defining different states of matter and energy and using known relations between them - figuring out what happens next. There is of course also an entire field studying what happens when we are not able to define the states well - non-equilibrium thermodynamics, but that is not for the faint of heart.

The name thermodynamics is old - from the time of Lord Kelvin, Lavosier and Lagrange and that bunch. Back then, you studied heat - where does heat go? If you burn something, what is the heat generated? How much capacity for heat does a substance have? How much hotter does it get with a fixed amount of heat? We inherit a lot from these old geezers, they were quite bright. We still call energies related to chemical reactions "Heat of Formation" - although most people use the word "Enthalpy". There are many of these "Heats" - sublimation, solidification, mixing... The study was thus called Thermo-dynamics - the movement of heat!