Adding a new and removing an old marker every time the user click on the map

Try something like this. Put the marker to a variable, and when creating a new one, it the variable already has a value remove it from the map, then draw the new marker.

 var theMarker = {};

  map.on('click',function(e){
    lat = e.latlng.lat;
    lon = e.latlng.lng;

    console.log("You clicked the map at LAT: "+ lat+" and LONG: "+lon );
        //Clear existing marker, 

        if (theMarker != undefined) {
              map.removeLayer(theMarker);
        };

    //Add a marker to show where you clicked.
     theMarker = L.marker([lat,lon]).addTo(map);  
});

I think the simplest way is to make the Marker as a global variable and change its coordinates instead of deleting it: Marker.setLatLng(lat, lng);