psql command "Error: PostgreSQL version 12 is not installed"

I've reproduced your case and here's how I resolved it:

Use sudo apt purge postgresql-12 answering Yes if you want to remove it completely:

  ┌───────────────────────────────────────────────────────────────────┤ Configuring postgresql-11 ├────────────────────────────────────────────────────────────────────┐
  │                                                                                                                                                                    │ 
  │ Removing the PostgreSQL server package will leave existing database clusters intact, i.e. their configuration, data, and log directories will not be removed. On   │ 
  │ purging the package, the directories can optionally be removed.                                                                                                    │ 
  │                                                                                                                                                                    │ 
  │ Remove PostgreSQL directories when package is purged?                                                                                                              │ 
  │                                                                                                                                                                    │ 
  │                                                  <Yes>                                                     <No>                                                    │ 
  │                                                                                                                                                                    │ 
  └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ 

After this:

sudo psql --version
psql (PostgreSQL) 11.7 (Ubuntu 11.7-2.pgdg19.10+1)