How to clean up Java ThreadLocals in accordance with Sonar?

Sonar is right here.

Each thread will have its own ThreadLocal state and so its own instance of NumberFormat.
So in the general case it may be undesirable to not clear data from the state since the thread may be reused (recycled by the server) and the state valued for the previous client may be inconsistent for the current client.
For example some clients could have the format US, others the format FR, and so for... Besides some threads could instantiate that ThreadLocal class, other no. But by not cleaning the state, the state will be still use memory for threads that may not need them.

Well, in your code, there is not variability of the ThreadLocal state since you set the state for any instance, so no inconsistency risk is likely, just memory "waste".

Now, I adopted the ThreadLocal approach in order to reuse NumberFormat instances as much as possible, avoiding the creation of one instance per call

You reuse the ThreadLocal state by a thread request basis.
So if you have 50 threads, you have 50 states.
In web applications, the server maps the client HTTP request to one thread.
So you don't create multiple instances of the formatter only in the scope of 1 http request. It means that If you use the formatter one or two time by request processing, the ThreadLocal cache doesn't bring a great value. But if you use it more, using it makes sense.

so I think if I called remove() somewhere in the code, I would lose all the advantages of this solution

Calling remove() will not hurt performance if you do that when the request processing is done. You don't lose any advantage since you may use the formatter dozen of times in the scope of the request and it will be cleaned only at the end.

You have Request Listener in the servlet specification : https://docs.oracle.com/javaee/7/api/javax/servlet/ServletRequestListener.html.
You could do that in void requestDestroyed(ServletRequestEvent sre).

You should not call #remove directly after you used the formatter. As you wrote that would defeat the purpose.

You only need to call #remove in the following case. Your web application is unloaded from the application server e.g. Tomcat. But the application server itself keeps on running.

In this case the application server probably keeps the threads it created for your application around and these threads will still have each an instance of NumberFormat associated with them. That's your memory leak.

So if you always restart the entire app server you probably don't need to care about this problem.

If you want to clean up the ThreadLocal properly you would want to call #remove once your application is starting to shut down. This way you reused the NumberFormat instance a maximum of times while still cleaning up properly.