I have a middleware where I a want to log every request/response. How can I access to POST data?

Here is complete solution I made

"""
Api middleware module
"""
import logging

request_logger = logging.getLogger('api.request.logger')


class LoggingMiddleware(object):
    """
    Provides full logging of requests and responses
    """
    _initial_http_body = None

    def process_request(self, request):
        self._initial_http_body = request.body # this requires because for some reasons there is no way to access request.body in the 'process_response' method.


    def process_response(self, request, response):
        """
        Adding request and response logging
        """
        if request.path.startswith('/api/') and \
                (request.method == "POST" and
                         request.META.get('CONTENT_TYPE') == 'application/json'
                 or request.method == "GET"):
            request_logger.log(logging.DEBUG,
                               "GET: {}. body: {} response code: {}. "
                               "response "
                               "content: {}"
                               .format(request.GET, self._initial_http_body,
                                       response.status_code,
                                       response.content), extra={
                    'tags': {
                        'url': request.build_absolute_uri()
                    }
                })
        return response

Note, this

'tags': {
    'url': request.build_absolute_uri()
}

will allow you to filter by url in sentry.


Andrey's solution will break on concurrent requests. You'd need to store the body somewhere in the request scope and fetch it in the process_response().

class RequestLoggerMiddleware(object):

    def process_request(self, request):
        request._body_to_log = request.body

    def process_response(self, request, response):
        if not hasattr(request, '_body_to_log'):
            return response

        msg = "method=%s path=%s status=%s request.body=%s response.body=%s"
        args = (request.method,
                request.path,
                response.status_code,
                request._body_to_log,
                response.content)

        request_logger.info(msg, *args)

        return response

All answers above have one potential problem -- big request.body passed to the server. In Django request.body is a property. (from framework)

@property
def body(self):
    if not hasattr(self, '_body'):
        if self._read_started:
            raise RawPostDataException("You cannot access body after reading from request's data stream")
        try:
            self._body = self.read()
        except IOError as e:
            six.reraise(UnreadablePostError, UnreadablePostError(*e.args), sys.exc_info()[2])
        self._stream = BytesIO(self._body)
    return self._body

Django framework access body directly only in one case. (from framework)

elif self.META.get('CONTENT_TYPE', '').startswith('application/x-www-form-urlencoded'):

As you can see, property body read the entire request into memory. As a result, your server can simply crash. Moreover, it becomes vulnerable to DoS attack. In this case I would suggest using another method of HttpRequest class. (from framework)

def readlines(self):
    return list(iter(self))

So, you no longer need to do this

def process_request(self, request):
    request._body_to_log = request.body

you can simply do:

def process_response(self, request, response):

    msg = "method=%s path=%s status=%s request.body=%s response.body=%s"
    args = (request.method,
            request.path,
            response.status_code,
            request.readlines(),
            response.content)

    request_logger.info(msg, *args)

    return response

EDIT: this approach with request.readlines() has problems. Sometimes it does not log anything.

Tags:

Rest

Django