How to ignore nulls while unmarshalling a MongoDB document?
The problem is that the current bson codecs do not support encoding / decoding string
into / from null
.
One way to handle this is to create a custom decoder for string
type in which we handle null
values: we just use the empty string (and more importantly don't report error).
Custom decoders are described by the type bsoncodec.ValueDecoder
. They can be registered at a bsoncodec.Registry
, using a bsoncodec.RegistryBuilder
for example.
Registries can be set / applied at multiple levels, even to a whole mongo.Client
, or to a mongo.Database
or just to a mongo.Collection
, when acquiring them, as part of their options, e.g. options.ClientOptions.SetRegistry()
.
First let's see how we can do this for string
, and next we'll see how to improve / generalize the solution to any type.
1. Handling null
strings
First things first, let's create a custom string decoder that can turn a null
into a(n empty) string:
import (
"go.mongodb.org/mongo-driver/bson/bsoncodec"
"go.mongodb.org/mongo-driver/bson/bsonrw"
"go.mongodb.org/mongo-driver/bson/bsontype"
)
type nullawareStrDecoder struct{}
func (nullawareStrDecoder) DecodeValue(dctx bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) error {
if !val.CanSet() || val.Kind() != reflect.String {
return errors.New("bad type or not settable")
}
var str string
var err error
switch vr.Type() {
case bsontype.String:
if str, err = vr.ReadString(); err != nil {
return err
}
case bsontype.Null: // THIS IS THE MISSING PIECE TO HANDLE NULL!
if err = vr.ReadNull(); err != nil {
return err
}
default:
return fmt.Errorf("cannot decode %v into a string type", vr.Type())
}
val.SetString(str)
return nil
}
OK, and now let's see how to utilize this custom string decoder to a mongo.Client
:
clientOpts := options.Client().
ApplyURI("mongodb://localhost:27017/").
SetRegistry(
bson.NewRegistryBuilder().
RegisterDecoder(reflect.TypeOf(""), nullawareStrDecoder{}).
Build(),
)
client, err := mongo.Connect(ctx, clientOpts)
From now on, using this client
, whenever you decode results into string
values, this registered nullawareStrDecoder
decoder will be called to handle the conversion, which accepts bson null
values and sets the Go empty string ""
.
But we can do better... Read on...
2. Handling null
values of any type: "type-neutral" null-aware decoder
One way would be to create a separate, custom decoder and register it for each type we wish to handle. That seems to be a lot of work.
What we may (and should) do instead is create a single, "type-neutral" custom decoder which handles just null
s, and if the BSON value is not null
, should call the default decoder to handle the non-null
value.
This is surprisingly simple:
type nullawareDecoder struct {
defDecoder bsoncodec.ValueDecoder
zeroValue reflect.Value
}
func (d *nullawareDecoder) DecodeValue(dctx bsoncodec.DecodeContext, vr bsonrw.ValueReader, val reflect.Value) error {
if vr.Type() != bsontype.Null {
return d.defDecoder.DecodeValue(dctx, vr, val)
}
if !val.CanSet() {
return errors.New("value not settable")
}
if err := vr.ReadNull(); err != nil {
return err
}
// Set the zero value of val's type:
val.Set(d.zeroValue)
return nil
}
We just have to figure out what to use for nullawareDecoder.defDecoder
. For this we may use the default registry: bson.DefaultRegistry
, we may lookup the default decoder for individual types. Cool.
So what we do now is register a value of our nullawareDecoder
for all types we want to handle null
s for. It's not that hard. We just list the types (or values of those types) we want this for, and we can take care of all with a simple loop:
customValues := []interface{}{
"", // string
int(0), // int
int32(0), // int32
}
rb := bson.NewRegistryBuilder()
for _, v := range customValues {
t := reflect.TypeOf(v)
defDecoder, err := bson.DefaultRegistry.LookupDecoder(t)
if err != nil {
panic(err)
}
rb.RegisterDecoder(t, &nullawareDecoder{defDecoder, reflect.Zero(t)})
}
clientOpts := options.Client().
ApplyURI("mongodb://localhost:27017/").
SetRegistry(rb.Build())
client, err := mongo.Connect(ctx, clientOpts)
In the example above I registered null-aware decoders for string
, int
and int32
, but you may extend this list to your liking, just add values of the desired types to the customValues
slice above.