how to roll back a transaction happening between microservices?

This situation can be handled only with JTA global/distributed transactions. JTA is part of Java EE standard and can have various implementors. Atomikos is often tool of choice.

Here is good writeup from Dave Syer (Spring ecosystem contributor). It contain also working examples. It's little bit outdated, but still relevant. You can apply some more modern Spring abstractions on top of his examples.

I created few GitHub examples of JTA transactions for my book. Notice that there are errors simulated and transaction is spread across JMS and JDBC datasources.

But also bear in mind that JTA transactions across various data sources are slow, because of 2-phased commit algorithm involved. So often people try to avoid them and rather deal with inconsistencies somehow pragmatically.

Don't do distributed transactions.

For integration with your existing legacy system one approach could be a separate (micro)service which listens to update events from your userService and forwards the respective updates to the legalService. Spring integration may be suitable for such a task.

Cheers, Michael

Well if you read little bit about the subject in the internet, it is a big debacle point at the moment but there is one answer that everybody agrees on it, distributed transactions are not way to go for it. They are too clumsy and buggy that we can't rely on them for data consistency.

So what is our options then, people are the moment trying to coordinate micro service transactions via Apache Kafka or with Event Source (which concentrate on saving events which are changing the data instead of saving the data itself). So what is the problem with those? Well they are quite different then usual programming model that we get used to and at technical and organisational point of view quite complex, so instead of programming for Business Problems, you start programming against the technical challenge.

So what is the alternative, I personally developed an another concept and wrote a blog about it, it might be interesting for you. In its basics, it uses full micro service design principles and Spring Boot + Netflix in a J2EE container and fully using the transactions, it is too long to write all details here, if you are interested you can read the from the link below.

Micro Services and Transactions with Spring Boot + Netflix