Xfce4 panel not responding to mouse(touch pad) clicks

I always got the same problem recently with xubuntu 14.04 My temporary solution is to type xfce4-panel -r in a terminal window to restart the panel.