Flask App Memory Leak caused by each API call

Important Note

Since this question was asked, Sanked Patel gave a talk at PyCon India 2019 about how to fix memory leaks in Flask. This is a summary of his strategy.

Minimal Example

Suppose you have a simple stateless Flask app with only one endpoint named 'foo'. Note that the other endpoints 'memory' and 'snapshot' aren't part of the original app. We need them later to find the memory leak.

import gc
import os
import tracemalloc

import psutil
from flask import Flask

app = Flask(__name__)
global_var = []
process = psutil.Process(os.getpid())
tracemalloc.start()
s = None


def _get_foo():
    global global_var
    global_var.append([1, "a", 3, True] * 10000)  # This is our (amplified) memory leak
    return {'foo': True}


@app.route('/foo')
def get_foo():
    gc.collect()  # does not help
    return _get_foo()


@app.route('/memory')
def print_memory():
    return {'memory': process.memory_info().rss}


@app.route("/snapshot")
def snap():
    global s
    if not s:
        s = tracemalloc.take_snapshot()
        return "taken snapshot\n"
    else:
        lines = []
        top_stats = tracemalloc.take_snapshot().compare_to(s, 'lineno')
        for stat in top_stats[:5]:
            lines.append(str(stat))
        return "\n".join(lines)


if __name__ == '__main__':
    app.run()

The memory leak is in line 17 and indicated by comment. Unfortunately, this is seldom the case. ;)

As you can see I have tried to fix the memory leak by calling garbage collection manually, i.e. gc.collect(), before returning a value at the endpoint 'foo'. But this doesn't solve the problem.

Finding the Memory Leak

To find out if there is a memory leak, we call the endpoint 'foo' multiple times and measure the memory usage before and after the API calls. Also, we will take two tracemalloc snapshots. tracemalloc is a debug tool to trace memory blocks allocated by Python. It is in the standard library if you use Python 3.4+.

The following script should clarify the strategy:

    import requests

    # Warm up, so you don't measure flask internal memory usage
        for _ in range(10):
        requests.get('http://127.0.0.1:5000/foo')

    # Memory usage before API calls
    resp = requests.get('http://127.0.0.1:5000/memory')
    print(f'Memory before API call {int(resp.json().get("memory"))}')

    # Take first memory usage snapshot
    resp = requests.get('http://127.0.0.1:5000/snapshot')

    # Start some API Calls
    for _ in range(50):
        requests.get('http://127.0.0.1:5000/foo')

    # Memory usage after
    resp = requests.get('http://127.0.0.1:5000/memory')
    print(f'Memory after API call: {int(resp.json().get("memory"))}')

    # Take 2nd snapshot and print result
    resp = requests.get('http://127.0.0.1:5000/snapshot')
    pprint(resp.text)

Output:

Memory before API call 35328000
Memory after API call: 52076544
('.../stackoverflow/flask_memory_leak.py:17: '
 'size=18.3 MiB (+15.3 MiB), count=124 (+100), average=151 KiB\n'
 '...\\lib\\tracemalloc.py:387: '
 'size=536 B (+536 B), count=3 (+3), average=179 B\n'
 '...\\lib\\site-packages\\werkzeug\\wrappers\\base_response.py:190: '
 'size=512 B (+512 B), count=1 (+1), average=512 B\n'
 '...\\lib\\tracemalloc.py:524: '
 'size=504 B (+504 B), count=2 (+2), average=252 B\n'
 '...\\lib\\site-packages\\werkzeug\\datastructures.py:1140: '
 'size=480 B (+480 B), count=1 (+1), average=480 B')

There is a large difference in memory usage before versus after the API calls, i.e. a memory leak. The second call of the snapshot endpoint returns the five highest memory usage differences. The first result locates the memory leak correctly in line 17.

If the memory leak hides deeper in the code, you may have to adapt the strategy. I have only scratched the capabilities of tracemalloc. But with this strategy you have a good starting point.

This behavior only happened to me in debug mode in development environment, but when I use waitress as web server my flask application works fine without memory leak.

This is my app.waitress to run from virtual environment.

import sys
import os
import site
from waitress import serve
dir_path = os.path.dirname(__file__)
sys.path.append(os.path.abspath(dir_path))
venv_packages =  os.path.abspath(os.path.join(dir_path, 'venv', 'lib', 'site-packages'))
sys.path.append(venv_packages)
site.addsitedir(venv_packages)
from dotenv import load_dotenv
dotenv_path = os.path.join(os.path.dirname(__file__), '.env')
load_dotenv(dotenv_path)
from settings import API_HOST, API_PORT
from app import app as application
serve(application, host=API_HOST, port=API_PORT)

To run it from terminal (Mac or Linux):

. venv/bin/activate
pip install waitress
python app.waitress

To run it from Windows:

py -3 -m pip install waitress
py app.waitress

Environment:

Python 3.7.9
waitress 1.4.1
Flask 1.1.2
Flask-Cors 3.0.10
Flask-JWT-Extended 3.25.0
python-dotenv 0.10.3