Does SNI represent a privacy concern for my website visitors?

Your analysis is incorrect. You are more secure with SNI than without.

Without SNI, the IP address uniquely identifies the host. Thus anyone who can determine the IP address can determine the host.

With SNI, the IP address does not uniquely identify the host. Someone would have to actually intercept and view some of the traffic to determine the exact host. This is more difficult than just obtaining the IP address.

So you are (slightly) more secure with SNI than without it.

Anyone who is going to block based on an intrusive analysis of packet data is going to also block based on IP address. They will block the "bad ones" based on IP address with or without SNI.

However, the answer to your question is "yes". SNI does represent a privacy concern. With SNI, someone who can intercept the traffic does get the host name in addition to the IP address.