Clock drift even though NTPD running
NTP daemons don't want to do a sudden massive jump in system time. For one, it borks the chronology of entries in your log files, system daemons might freak out, etc. What it does instead is "drift" your system clock into place. If you want to check how far off from "the actual time" you are you can query an ntp server:
ntpdate -q pool.ntp.org
NTP works by adjusting the length of a second on your system by a slight bit so that you slowly get the correct time. It can take a while for the drift to happen if your offset is high. What you can do though is force a hard sync once:
service ntp stop
ntpdate pool.ntp.org
service ntp start
Edit: in response to the comments below.
Set this line up in /etc/default/ntpdate
NTPDATE_USE_NTP_CONF=yes
There have been some problems with NTP synchronisation in Ubuntu. Some fixes were identified and will be in the 11.04 release in April.
Colin Watson wrote a detailed blog post about this as a response to an Ubuntu Brainstorm post.