How can I bypass the "too expensive" cap for the anvil in Minecraft?

What you're describing is an old bug - however there are still mechanics that cover this.

According to the minecraft gampedia renaming an item will not increase it's cost to do something else to it, however if you rename and do somethig else, to it, it's cost will still increase as if you were not naming it at all.

In survival, you can only work an item 6 times, including repairs (but as I stated not including renames alone), with the xp cost increasing each time.

In creative, you can work it 31 times before even then you will be unable to work it again (noting that if you work it this many time you will loose the ability to even rename it).

It's worth noting currently, the only vanilla way to reset this xp multiplier is to merge it with another damaged tool in a standard crafting table, however this will remove all enchants. Repairing it with another tool in an anvil will not work as that will take the highest xp value.