Installing latest version of GNU Octave when I have already installed other version

For uninstalling GNU Octave you can run this command:

sudo apt purge octave

But you don't need to remove and then install latest version of Octave.

You can easily upgrade that package with this commands:

sudo add-apt-repository ppa:octave/stable
sudo apt update
sudo apt install octave

These commands upgrade Octave to latest stable version of this software (with no need to remove current version).

You can run the newest version of Octave (5.1.0 in March 2019) by running it as a Flatpack app. "Flatpack is a system for building, distributing, and running sandboxed desktop applications on Linux."

The installation procedure is described here: https://flathub.org/apps/details/org.octave.Octave at the bottom of the page.

A more complete set of commands looks like the following section:

If you are on Ubuntu 16.04, You have to install flatpack first, then run Octave:

sudo add-apt-repository ppa:alexlarsson/flatpak
sudo apt install flatpak
# optional line, AFAIK (pulls in Gnome deps?):
sudo apt install gnome-software-plugin-flatpak  
# this line possibly required on Debian systems
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo  
flatpak install flathub org.octave.Octave
flatpak run org.octave.Octave --gui

This downloads about 900 MB from the internet.

For convenience in starting Octave from the command line, you can create the files /usr/share/octave with the contents

#!/bin/sh
flatpak run org.octave.Octave --gui

and /usr/share/octave-cli with content

#!/bin/sh
flatpak run org.octave.Octave