apt remove vs purge

You can read the manual of apt with

man apt

On line 35 you will find the following

       Removing a package removes all packaged data, but leaves usually
       small (modified) user configuration files behind, in case the
       remove was an accident. Just issuing an installation request for
       the accidentally removed package will restore its function as
       before in that case. On the other hand you can get rid of these
       leftovers by calling purge even on already removed packages. Note
       that this does not affect any data or configuration stored in your
       home directory.

So in short: remove leaves some config files behind where purge does not. And you can even use purge if later you want to delete those config files.

The manpage of apt-get (run man apt-get to see this) says:

remove
    remove is identical to install except that packages are removed instead of installed. Note that removing a
    package leaves its configuration files on the system. If a plus sign is appended to the package name (with
    no intervening space), the identified package will be installed instead of removed.

purge
    purge is identical to remove except that packages are removed and purged (any configuration files are
    deleted too).

In a nutshell, remove keeps the configuration files while purge removes them. Both are safe, but which one to pick depends on whether you want to remove configurations.