Is it rude to change universities/supervisors after already indicating research interest with a current supervisor?

If Y is what you really want to do then go for it. There is no point in going down a career path just to please someone else.

Talk in person to your current supervisor once you have a written offer from the university that does Y. Explain to him that it is your dream to do Y and you hope to keep in contact with him even though you are leaving the university.

A good supervisor will understand - if not then he was not a good supervisor anyway. It's a win-win for you.

If you have spent the time to earn a degree somewhere and have interacted with the faculty there who are closest to your interests, then it is actually a good thing to move on to another university for continued education. Don't get too comfortable early on.

You have learned a lot from a few people, but other people have different ways to think and it is good to get exposed to that early in your career.

Many, maybe most, faculty understand this. Especially undergraduate faculty. I always thought it was best if my best students left me behind and found inspiration elsewhere. My view is that it is much better if my students can surpass me than if they wind up just like me.

So, don't worry much about rudeness as much as you worry about maximizing your own opportunities. Thank your professor for all that s/he has done for you and for making it possible for you to move on. In a few years, return and thank them again and bring them up to date on your success.