Is it bad idea to do apt-get upgrade in OpenVZ container?

Assuming the server setup is sane and you are upgrading within a stable release (upgrading to a new major version or following a development release is a different thing) then apt-get upgrade should not normally be a problem and needs to be done to get security fixes.

It sounds like your hosting provider has screwed up big time on preparing their images. If they can't find a soloution that allows you to apply normal distribution updates to your server then it;s time to vote with your feet.