Thread-safe HashSet with Guava Collections
Set<K> set = Collections.newSetFromMap(new ConcurrentHashMap<K, Boolean>());
This would be the right answer, Using the Sets class from Guava. Anyway the answer from @crhis was good intended.
Sets.newSetFromMap(new ConcurrentHashMap<V, Boolean>());