Can my domain registrar requests high price for renewal if my domain became famous?

The first reason is competition. The registrar cannot force you to stay with them and there are tons of registrars. If they charged you 10 times their public price you'd just go to the next one.

The second reason is economics. If your domain is famous, you probably need a lot of other stuff : emails, servers, IPs... If they lost all their big customers, it would be a terrible business strategy. Doesn't make sense economically.

The third reason is practicality. A good registrar will manage hundreds of thousands of domain. It is very hard to keep track and know which domains actually drive good traffic. What is a "famous" domain? It's hard to define this programmatically. And there's no point in doing it by hand to charge 100€ for its renewal.

Besides the fact that your question may depend on the TLD, the answer is "probably nothing", except that if this practice becomes known I think it would attract bad buzz and hence clients going elsewhere.

Also as comments hinted at, you are free to transfer your domain to another registrar at any time (after 60 days) and since you are also free to renew at any time (at least for all gTLDs and most ccTLDs), you can easily see if there are problems on the price. Again, in gTLD world and many ccTLD you can register or renew your domain for up to 10 years, which can shield you quite a long time from price changes: if your domain is famous or even if it is not but at least as soon as it is the basis of some business, it makes no sense to renew it from one year to another, it should be registered for at least 5 years in advance.

Note however that registries, at least in gTLD world are mostly free to change their price at any time and obviously these changes of price will get passed on by registrars to end clients. Things also depend if the domain was a premium from the beginning or not: some premiums will renew at standard price, some as premium registration price, this all differs from one registry, or TLD, to the other.

Also, a recent change of contract for .ORG basically allows the registry to no set prices per domain, if it so wishes (their communication for now is that they won't do that). See http://domainincite.com/24458-org-now-has-no-price-caps-but-no-specific-plans-to-raise-prices for example for some details on this.