When should Flask.g be used?

As an addendum to the information in this thread: I've been a bit confused by the behavior of flask.g too, but some quick testing has helped me to clarify it. Here's what I tried out:

from flask import Flask, g
app = Flask(__name__)

with app.app_context():
    print('in app context, before first request context')
    print('setting g.foo to abc')
    g.foo = 'abc'
    print('g.foo should be abc, is: {0}'.format(g.foo))

    with app.test_request_context():
        print('in first request context')
        print('g.foo should be abc, is: {0}'.format(g.foo))
        print('setting g.foo to xyz')
        g.foo = 'xyz'
        print('g.foo should be xyz, is: {0}'.format(g.foo))

    print('in app context, after first request context')
    print('g.foo should be abc, is: {0}'.format(g.foo))

    with app.test_request_context():
        print('in second request context')
        print('g.foo should be abc, is: {0}'.format(g.foo))
        print('setting g.foo to pqr')
        g.foo = 'pqr'
        print('g.foo should be pqr, is: {0}'.format(g.foo))

    print('in app context, after second request context')
    print('g.foo should be abc, is: {0}'.format(g.foo))

And here's the output that it gives:

in app context, before first request context
setting g.foo to abc
g.foo should be abc, is: abc  

in first request context
g.foo should be abc, is: abc
setting g.foo to xyz
g.foo should be xyz, is: xyz  

in app context, after first request context
g.foo should be abc, is: xyz  

in second request context
g.foo should be abc, is: xyz
setting g.foo to pqr
g.foo should be pqr, is: pqr  

in app context, after second request context
g.foo should be abc, is: pqr

As theY4Kman said above, "Every request pushes a new application context". And as the Flask docs say, the application context "will not be shared between requests". Now, what hasn't been explicitly stated (although I guess it's implied from these statements), and what my testing clearly shows, is that you should never explicitly create multiple request contexts nested inside one application context, because flask.g (and co) doesn't have any magic whereby it functions in the two different "levels" of context, with different states existing independently at the application and request levels.

The reality is that "application context" is potentially quite a misleading name, because app.app_context() is a per-request context, exactly the same as the "request context". Think of it as a "request context lite", only required in the case where you need some of the variables that normally require a request context, but you don't need access to any request object (e.g. when running batch DB operations in a shell script). If you try and extend the application context to encompass more than one request context, you're asking for trouble. So, rather than my test above, you should instead write code like this with Flask's contexts:

from flask import Flask, g
app = Flask(__name__)

with app.app_context():
    print('in app context, before first request context')
    print('setting g.foo to abc')
    g.foo = 'abc'
    print('g.foo should be abc, is: {0}'.format(g.foo))

with app.test_request_context():
    print('in first request context')
    print('g.foo should be None, is: {0}'.format(g.get('foo')))
    print('setting g.foo to xyz')
    g.foo = 'xyz'
    print('g.foo should be xyz, is: {0}'.format(g.foo))

with app.test_request_context():
    print('in second request context')
    print('g.foo should be None, is: {0}'.format(g.get('foo')))
    print('setting g.foo to pqr')
    g.foo = 'pqr'
    print('g.foo should be pqr, is: {0}'.format(g.foo))

Which will give the expected results:

in app context, before first request context
setting g.foo to abc
g.foo should be abc, is: abc  

in first request context
g.foo should be None, is: None
setting g.foo to xyz
g.foo should be xyz, is: xyz  

in second request context
g.foo should be None, is: None
setting g.foo to pqr
g.foo should be pqr, is: pqr

Advanced Flask Patterns, as linked by Markus, explains some of the changes to g in 0.10:

• g now lives in the application context.
• Every request pushes a new application context, wiping the old one, so g can still be used to set flags per-request without change to code.
• The application context is popped after teardown_request is called. (Armin's presentation explains this is because things like creating DB connections are tasks which setup the environment for the request, and should not be handled inside before_request and after_request)