Is this the cleanest way to write a long list in Python?

You need to indent the list contents like this

kitchen_items = [
    "Rice", "Chickpeas", "Pulses", "bread", "meat",
    "Milk", "Bacon", "Eggs", "Rice Cooker", "Sauce",
    "Chicken Pie", "Apple Pie", "Pudding"
]

Or

kitchen_items = [
    "Rice", "Chickpeas", "Pulses", "bread", "meat",
    "Milk", "Bacon", "Eggs", "Rice Cooker", "Sauce",
    "Chicken Pie", "Apple Pie", "Pudding"
    ]

The section you quoted:

The closing brace/bracket/parenthesis on multi-line constructs may either line up under the first non-whitespace character of the last line of list

Honestly, it means exactly what it says:

my_list = [
    'a', 'b', 'c', 'd',
    'e', 'f', 'g', 'h',  <-- "the last line of the list"
    ^
    "the first non-whitespace character"

Thus:

my_list = [
    'a', 'b', 'c', 'd',
    'e', 'f', 'g', 'h',
    ]

There's also the second option that PEP-8 refers to,

or it may be lined up under the first character of the line that starts the multi-line construct, as in:

"the first character"
v
my_list = [  <-- "line that starts the multi-line construct"
    'a', 'b', 'c', 'd',
    'e', 'f', 'g', 'h',

Thus:

my_list = [
    'a', 'b', 'c', 'd',
    'e', 'f', 'g', 'h',
]

Personally, I prefer this second style, because it give a nice way to scan for the end of the list: the ] justs back out into the left-hand side:

my_list = [
|    'items', 'items',
|    'items', 'items',
|  < a nice line for your eye to track
|
|
]  < this stands out more