If a lithium cell is said to have a capacity of 20 Ah, does that mean all 20 Ah can be used?
The available capacity of a fully charged battery varies with temperature and discharge rate (and duty cycle if it's not constant). It will tend to decrease with each cycle, and previous fast charging, exposure to temperature extremes or fast discharging may have negative effects over time.
If a reputable manufacturer says it is rated at 20Ah that means that under the specified conditions you can get approximately 20Ah from a new battery before it discharges to the point where further discharge may cause damage. The details of how it behaves should be described in the datasheet and application literature. You may wish, for example, to charge slower than the maximum or discharge only to a higher voltage than the minimum in order to extend the battery life. The latter will, of course, reduce the available capacity.
If you'll forgive an analogy, your question is a bit like asking about a car manufacturer that claims 7 liters per 100km (33.6 miles per US gallon) fuel efficiency and whether that means you'll actually get that efficiency. It depends on a lot of factors, such as your driving habits and the condition of the automobile.
The available current capacity is listed in the data sheet. This is typically listed as current over time at a specific discharge rate, from fully charged to the dictated cut off voltage.
If the operating voltage range is 2.7V to 1.5V, then the stated available capacity when new should be within that range.
Batteries still have nonzero voltage when they are completely discharged and can supply no more current. They do not discharge down to zero volts. An empty battery reads higher than 0V. It's not like the fluid level in a gas tank.
Therefore, all 20Ah are between 1.5V and 2.7V. But even then you can still only use about 80% of that without damaging the battery. If it drains so far that it gets to 1.5V then the battery already is completely discharged (and probably damaged too from being discharged so deeply).