Is set returned by map keyset immutable?

A Set from a Map does not support Set.add or Set.addAll. The Set itself is not immutable, but it throws an Exception if you try to add values, because it would cause the Map to have null values for the new keys.

As an example of mutability, the following code is valid:

Map<Integer, String> v = new Map<Integer, String> { 1 => 'hello', 2 => 'world' };
Set<Integer> a = new Set<Integer> { 1 };
v.keySet().retainAll(a); // v now contains just one key/value.

As mentioned by Adrian, if you want to combine two key sets, you need a normal Set that can be modified.

If you want to combine two keysets, you can clone them first, or simpler yet, initialize your collection with an empty set first:

Set<Integer> keys = new Set<Integer>();
keys.addAll(m1.keySet());
keys.addAll(m2.keySet());

Objects in apex are passed by reference, so when you assign Set<Integer> s = m1.keyset(); you get reference to read-only collection of keys of Map. So you need to have new Set, that can be changed

map<integer, integer> m1 = new Map<integer, integer>{1=>1,2=>2};
map<integer, integer> m2 = new Map<integer, integer>{3=>3,4=>4};
Set<Integer> s = new Set<Integer>(m1.keyset());
s.addAll(m2.keySet());

Non-primitive data type arguments, such as sObjects, are passed into methods by reference. This means that when the method returns, the passed-in argument still references the same object as before the method call and can't be changed to point to another object. However, the values of the object's fields can be changed in the method.

this is from here