Clarifying the actual definition of elasticity. Is steel really more elastic than rubber?
There are two separate concepts here:
the Young's modulus, which determines the force needed to stretch the material
the elastic limit, aka yield strain, which determines how far the material can be stretched
As you say, the term elastic tends to be used in a vague way that conflates these two properties. Generally a high Young's modulus means the material is stiff so I would say steel is stiffer than rubber not more elastic than rubber. Steel also has a much smaller yield strain that rubber because you can't stretch steel far before it starts to deform while rubber can be stretched a long distance.
So if you're going to use the vaguely defined term elastic then steel is certainly less elastic than rubber in both meanings. However in a physics or engineering context you would use the precisely defined terms Young's modulus and yield strain instead.
Finally:
There is another meaning for elastic, which is what Rod has covered in his answer. I'm going to summarise it here for completeness but please upvote Rod's answer as he thought of it first!
If we say a collision is elastic it means no energy is lost in the collision. In this sense the collision between steel balls is highly elastic. That's why a Newton's cradle with steel balls will swing for ages once you set it going. By contrast collisions between rubber balls tend to be squidgier and lose more energy so in this sense they are less elastic than steel. It might be that this is why you have seen steel described as more elastic than rubber. The term elastic applies to the collision rather than the material.
Both the OP and John Rennie have well illustrated the imperfections in the usage of the word "elastic" in physics and how the word can create confusion between "stiffness" and a material's ability to brook strain.
But an important point to be made is that the one important field where one hears the vague statement that "steel is more elastic than rubber" is in the context of Newtonian collision problems. So what's meant here is that steel objects typically undergo more elastic, i.e. kinetic energy conserving, collisions than rubber ones.
Newtonian collision problems come up very early and prominently in an undergraduate physics course, so this may well create the (probably mistaken) impression that physicists tend to mean stiffness rather than ability to brook strain by the word "elastic". Indeed, the fields wherein physicists, as opposed to specialist material scientists, mostly use the word "elastic" are those where the word refers to collisions and interactions, and in these contexts the word means "conserving of total kinetic energy of all colliding bodies" or "not energy converting". Elastic optical interactions such as Rayleigh scattering or Fresnel reflexions are those where the incident and scattered light have the same wavelength, thus photon energy, and no energy is dissipated in or transferred to the scatterer. Likewise with all particle physics specialities.
An interesting comment from user Jasper:
In other words, rubber's stress-strain curve has more hysteresis (as a fraction of the maximum strain energy in the loop) when the strain goes from negative to positive and back.
Intuitively, it's probably part of the cause, maybe the main cause in some materials but there are rubbers where other mechanisms account for the loss, according to some cursory research I've been doing into rubbers in recent weeks for bearings in an adaptive optics system I've been working on. I'm certainly no expert, but common models used are all linear differential equation models wherein the loss comes from damping terms. Look up the Kelvin Voigt Model and Maxwell-Wiechert Model and Standardized Linear Model. Synthetic rubber manufacturers often specify the loss properties of their wares by loss tangents and complex-valued Young's modulusses (which show a phase delay for sinusoidal force excitation). Mechanisms other than hysteresis that can give rise to loss tangents are viscous drag between neighboring molecules; this can be simply linear damping of the form $-\mu\,\dot{x}$ where $x$ measures the strain and $\mu$ a viscous drag term. To be clear: by "hysteresis" I mean a nonlinear, instantaneous two-valuedness of a strain-stress response curve where which of the two function branches is traversed is set by the direction of the variation. Each cycle around a $B\, vs.\,H$ loop in a ferromagnetic material or around a $\sigma\,vs.\,\epsilon$ loop in a deformable material transfers energy proportional to the loop area to the material. This is different from viscous drag.
My interpretation of the statement "steel is more elastic than rubber" is very different from yours.
I would say that it means rubber is viscoelastic and that there is a time dependence to the stress-strain relationship; it flows when you shear it. Steel will be very nearly perfectly elastic until reaching yield.
Understood this way, we can say that for a given stress OR strain applied, rubber will never be perfectly elastic. This is, by the way, basically equivalent to saying that no energy is lost in elastic collisions, as that energy is going into rearranging long chains of hydrocarbons in rubber instead of just vibrating an iron-carbon lattice and slightly heating it up.