Inserting a document with Pymongo - InvalidDocument: Cannot encode object
- If you have numpy object for ex. int or float in the json/dict data_dict which you want to send over mongo using pymongo.
- one might get "cannot encode object" error, to resolve this I have used a custom encoder like this.
class CustomEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, numpy.integer):
return int(obj)
elif isinstance(obj, numpy.floating):
return float(obj)
elif isinstance(obj, numpy.ndarray):
return obj.tolist()
else:
return super(CustomEncoder, self).default(obj)
data_dict_1 = json.dumps(data_dict,cls=CustomEncoder)
data_dict_final = json.loads(data_dict_1)
- Please check out docs here https://docs.python.org/3/library/json.html
- this way does not matter how your Json data is organised, it works.
Your problem is that numpy.int64
is foreign to MongoDB. I have had the same problem.
The solution is to convert the offending values to a datatype that MongoDB will understand, here is an example how I converted those offending values in my code:
try:
collection.insert(r)
except pymongo.errors.InvalidDocument:
# Python 2.7.10 on Windows and Pymongo are not forgiving
# If you have foreign data types you have to convert them
n = {}
for k, v in r.items():
if isinstance(k, unicode):
for i in ['utf-8', 'iso-8859-1']:
try:
k = k.encode(i)
except (UnicodeEncodeError, UnicodeDecodeError):
continue
if isinstance(v, np.int64):
self.info("k is %s , v is %s" % (k, v))
v = int(v)
self.info("V is %s" % v)
if isinstance(v, unicode):
for i in ['utf-8', 'iso-8859-1']:
try:
v = v.encode(i)
except (UnicodeEncodeError, UnicodeDecodeError):
continue
n[k] = v
collection.insert(n)
I hope this helps you.