Would floating a Lithium battery indefinitely at a lower voltage than max voltage cause any harm?

if we set 4.1V as CV phase for a Lithium Ion battery and keep it under charge indefinitely? Would it still cause any damage?

A Lithium Ion battery has limited lifespan at any voltage, but it degrades faster at higher voltage and temperature. So the question is, what amount of degradation do we call 'damage'?

In the graph below (from Aging of Lithium-Ion Batteries in Electric Vehicles by Peter Keil) we see that these particular cells (Panasonic NCR18650PD) show a distinct reduction in capacity after being stored at charge levels above 60%, but little difference between 88% (4.1 V) and 100% (4.2 V) charge at 25 °C. As storage temperature increases the degradation becomes more affected by state of charge, but variation at the high end is only significant well above normal ambient temperature.

Even though the voltage when continuously 'floated' would be a bit higher, this data suggests that lowering the voltage may not have much effect above ~3.8 V (60% charge) - at least for this type of cell. However it might be different for other cell chemistries.

Then you must balance the 'damage' against the loss of functionality at lower voltage. For longest life you should float below 3.8 V, but this provides less than 60% capacity. If having to top up before running on battery power is not acceptable then a slightly shorter lifespan is probably a small price to pay for not needing a battery with twice the rated capacity.

On the other hand, floating at 4.1 V more closely matches the long-term resting voltage of a fully charged cell. This may reduce the chances of catastrophic failure due to high voltage stress, while not greatly affecting the available capacity.