remove null character from string
You can remove \x00
runes from a string the same way you can remove any other runes:
valueStr = strings.Replace(valueStr, "\x00", "", -1)
Example:
s := "a\x00b"
fmt.Printf("%q\n", s)
s = strings.Replace(s, "\x00", "", -1)
fmt.Printf("%q\n", s)
Output (try it on the Go Playground):
"a\x00b"
"ab"
Using strings.Replacer
Also note that you can substitute the multiple replaces with a single operation by using strings.Replacer
, and it will also be more efficient as it only iterates over the input once (and there will be only one string
allocated for the result, no matter how many substrings you want to replace).
For example:
s := " \t\n\rabc\x00"
fmt.Printf("%q\n", s)
r := strings.NewReplacer(" ", "", "\t", "", "\n", "", "\r", "", "\x00", "")
s = r.Replace(s)
fmt.Printf("%q\n", s)
Output (try it on the Go Playground):
" \t\n\rabc\x00"
"abc"
Also note that it's enough to create a string.Replacer
once, and you can store it in a (global) variable and reuse it, it is even safe to use it concurrently from multiple goroutines.
Using strings.Map()
Also note that if you only want to replace (remove) single rune
s and not multi-rune (or multi-byte) substrings, you can also use strings.Map()
which might be even more efficient than strings.Replacer
.
First define a function that tells which rune
s to replace (or remove if you return a negative value):
func remove(r rune) rune {
switch r {
case ' ', '\t', '\n', '\r', 0:
return -1
}
return r
}
And then using it:
s := " \t\n\rabc\x00"
fmt.Printf("%q\n", s)
s = strings.Map(remove, s)
fmt.Printf("%q\n", s)
Output (try it on the Go Playground):
" \t\n\rabc\x00"
"abc"
Benchmarks
We might think strings.Map()
will be superior as it only have to deal with rune
s which are just int32
numbers, while strings.Replacer
have to deal with string
values which are headers (length+data pointer) plus a series of bytes.
But we should know that string
values are stored as UTF-8 byte sequences in memory, which means strings.Map()
have to decode the rune
s from the UTF-8 byte sequence (and encode the runes back to UTF-8 in the end), while strings.Replacer
does not: it may simply look for byte sequence matches without decoding the rune
s. And strings.Replacer
is highly optimized to take advantage of such "tricks".
So let's create a benchmark to compare them:
We'll use these for the benchmarks:
var r = strings.NewReplacer(" ", "", "\t", "", "\n", "", "\r", "", "\x00", "")
func remove(r rune) rune {
switch r {
case ' ', '\t', '\n', '\r', 0:
return -1
}
return r
}
And we run benchmarks on different input strings:
func BenchmarkReplaces(b *testing.B) {
cases := []struct {
title string
input string
}{
{
title: "None",
input: "abc",
},
{
title: "Normal",
input: " \t\n\rabc\x00",
},
{
title: "Long",
input: "adsfWR \t\rab\nc\x00 \t\n\rabc\x00asdfWER\n\r",
},
}
for _, c := range cases {
b.Run("Replacer-"+c.title, func(b *testing.B) {
for i := 0; i < b.N; i++ {
r.Replace(c.input)
}
})
b.Run("Map-"+c.title, func(b *testing.B) {
for i := 0; i < b.N; i++ {
strings.Map(remove, c.input)
}
})
}
}
And now let's see the benchmark results:
BenchmarkReplaces/Replacer-None-4 100000000 12.3 ns/op 0 B/op 0 allocs/op
BenchmarkReplaces/Map-None-4 100000000 16.1 ns/op 0 B/op 0 allocs/op
BenchmarkReplaces/Replacer-Normal-4 20000000 92.7 ns/op 6 B/op 2 allocs/op
BenchmarkReplaces/Map-Normal-4 20000000 92.4 ns/op 16 B/op 2 allocs/op
BenchmarkReplaces/Replacer-Long-4 5000000 234 ns/op 64 B/op 2 allocs/op
BenchmarkReplaces/Map-Long-4 5000000 235 ns/op 80 B/op 2 allocs/op
Despite expectations, string.Replacer
performs pretty good, just as good as strings.Map()
due to it not having to decode and encode runes.