Is it rude to visit a professor's office during office hours after he/she doesn't respond to my email?

I'm with your parents on this. Go talk to him. The missed reply could be for a variety of reasons, including just needing time to think about it. But scheduled office hours are a perfect time for a conversation.

It isn't rude at all. Polite, actually, to go in person.