Charging li-ion cell using constant-voltage only

(the numbers are given for LiCoO2 chemistry, for LiFePO4 they should be lower) Short answer: yes, this is OK. I charge lithium cells with a bench supply set to 4.2 V and whatever C/10 figure is for the cell I'm charging. If you are not in a hurry, slow charging is better, even for cells claimed to be capable of withstanding higher charging currents.

However (quoting you):

charging at a constant voltage (say 4.2V) so long as the maximum current is limited to a reasonable value for the cell

means you will have constant current charger till your cell is at ~95%. Up to this point the voltage across the battery will be less than 4.2V if you measure it. Only when your charger starts outputting 4.2V it will become constant voltage.

What you are about to build is CC/CV charger and this is the right thing to do. "Constant voltage only" charger will be set to 4.2 V with no current limiting and it will charge the lithium cell very slowly. You can check it youself, just construct var.voltage circuit and measure the current into (discharged) cell at 3.5, 3.7, 4.0, 4.2, and 4.5 V. Cheap Chinese chargers are constructed like that, they restrict the voltage to 4.2 V so the cell won't ignite after being charged but the consumer would have to wait longer. I once bought a portable emergency charger which could be emptied in a couple of hours then took 3 days to recharge its internal cells.

There are other precautions to observe while charging lithium, you can learn them from any modern charge controller IC datasheet (my favorite is Linear Tech., their literature is very high quality). If you don't implement these precautions in your design never leave it unattended while charging, otherwise it may ruin your morning one day.

That would be an excellent way to start a fire, so yes, DO limit current to manufacturer recommended values.

But to answer the question: no downside. I prefer to charge up to 4.1v to make my cells live longer but that's unrelated.

The key point you make is: "so long as the maximum current is limited to a reasonable value for the cell (say 0.5C)" That alone will make it so the 4.2v you aim for will be dropped so as to push only 0.5C in the battery at a given instant.

My experience: I actually tried to push a constant 4.2v with NO current limit through a li ion cell.

The amps can reach double digits and the cell hates it so much that it heats up like crazy.