removing all non-printing characters by regex
All the characters you provided belong to the Separator, space Unicode category, so, you may use
s = s.replaceAll("\\p{Zs}+", " ");
The Zs
Unicode category stands fro space separators of any kind (see more cateogry names in the documentation).
To replace all horizontal whitespaces with a single regular ASCII space you may use
s = s.replaceAll("\\h+", " ");
As per Java regex documentation,
\h
A horizontal whitespace character:[ \t\xA0\u1680\u180e\u2000-\u200a\u202f\u205f\u3000]
If you want to shrink all Unicode whitespace to a single space
s = s.replaceAll("(?U)\\s+", " ");
The (?U)
is an embedded flag option equal to the Pattern.UNICODE_CHARACTER_CLASS
option passed to the Pattern.compile
method. Without it, \s
matches what \p{Space}
matches, i.e. [ \t\n\x0B\f\r]
. Once you pass (?U)
, it will start matching all whitespace chars in the Unicode table.
To tokenize a string, you may split directly with
String[] tokens = s.split("\\p{Zs}+");
String[] tokens = s.split("\\h+");
String[] tokens = s.split("(?U)\\s+");
There is also a POSIX like [^[:graph:]]
available. For one or more non visible characters, try
\P{Graph}+
The upper P indicates a negation of \p{Graph}
and would match one or more [^\p{Alnum}\p{Punct}]
or [\p{Z}\p{C}]
. Downside is, that it's US-ASCII only according to the manual. If working with UTF-8 consider using inline flag (?U)
or UNICODE_CHARACTER_CLASS
.
Just to mention, there is further \P{Print}
available for non printable characters.