Drupal - What difference between declaration of an ajax callback function with $this and without it?

The ::method as a string is a special feature of the form API.

It's basically a shorthand for [$this, 'method'] and does almost exactly the same. See FormState::prepareCallback

There is one interesting difference, [$this, 'method'] has to serialize the whole form object within the form structure, to then unserialize it to call a method on it. But at that point, on a cached request, it is actually a different object because the form system also created a new form object, as part of initializing it and starting to process the request. ::method will actually call the method on the new form object.

Whenever possible, use the short version, sometimes that's not possible, for example when you are inside a plugin (e.g. a field widget) or other class called by the form, in this case, it is recommend to use a static method.

Its the difference between a static method call and an object method call. As per the PHP docs:

// Type 3: Object method call
$obj = new MyClass();
call_user_func(array($obj, 'myCallbackMethod'));

// Type 4: Static class method call (As of PHP 5.2.3)
call_user_func('MyClass::myCallbackMethod');

Your first callback is an object call to your method. The second one is a static call to a static method. Since PHP is a forgiving language both approaches will work (PHP will try to resolve them for you) but will emit warnings if your static callback will call an object method and vice versa.

In static context you are not able to call the object methods in your class. In object context however you are able to call static methods. I would recommend to use objects calls as much as possible.