Doing the equivalent of log_struct in python logger

Current solution:

Update 1 - user Seth Nickell improved my proposed solution, so I update this answer as his method is superior. The following is based on his response on GitHub:

https://github.com/snickell/google_structlog

pip install google-structlog

Used like so:

import google_structlog

google_structlog.setup(log_name="here-is-mylilapp")

# Now you can use structlog to get searchable json details in stackdriver...
import structlog
logger = structlog.get_logger()
logger.error("Uhoh, something bad did", moreinfo="it was bad", years_back_luck=5)

# Of course, you can still use plain ol' logging stdlib to get { "message": ... } objects
import logging
logger = logging.getLogger("yoyo")
logger.error("Regular logging calls will work happily too")

# Now you can search stackdriver with the query:
# logName: 'here-is-mylilapp'

---

Original answer:

Based on an answer from this GitHub thread, I use the following bodge to log custom objects as info payload. It derives more from the original _Worker.enqueue and supports passing custom fields.

from google.cloud.logging import _helpers
from google.cloud.logging.handlers.transports.background_thread import _Worker

def my_enqueue(self, record, message, resource=None, labels=None, trace=None, span_id=None):
    queue_entry = {
        "info": {"message": message, "python_logger": record.name},
        "severity": _helpers._normalize_severity(record.levelno),
        "resource": resource,
        "labels": labels,
        "trace": trace,
        "span_id": span_id,
        "timestamp": datetime.datetime.utcfromtimestamp(record.created),
    }

    if 'custom_fields' in record:
        entry['info']['custom_fields'] = record.custom_fields

    self._queue.put_nowait(queue_entry)

_Worker.enqueue = my_enqueue

Then

import logging
from google.cloud import logging as google_logging

logger = logging.getLogger('my_log_client')
logger.addHandler(CloudLoggingHandler(google_logging.Client(), 'my_log_client'))

logger.info('hello', extra={'custom_fields':{'foo': 1, 'bar':{'tzar':3}}})

Resulting in:

Which then makes it much easier to filter according to these custom_fields.

Let's admit this is not good programming, though until this functionality is officially supported there doesn't seem to be much else that can be done.