I have two versions of LibreOffice installed: how to remove the older one?

Try this command:

sudo apt-get remove --purge libreoffice4.2*

What this does is matching of the libreoffice packages with the version 4.2 and removes them. After removing it is advised to reboot.

Source: http://www.askmetutorials.com/2014/03/install-uninstall-libreoffice-422-on.html

If the "older" version that you want to remove is different than 4.2 or if you want to have a more recent version than the one you have now, you may consider using Synaptic Package Manager to remove all the versions that you installed (searching 'libreoffice') and then

 sudo add-apt-repository ppa:libreoffice/ppa
 sudo apt-get update
 sudo apt-get install libreoffice

I always install libreoffice using the .deb files. When I need to remove an older libreoffice version I use Synaptic. Go to Synaptic, search for libreoffice, right click on libreofficeX.y-core (X and y is your older version), click on Mark for Complete Removal, click on the Apply button and it is done!