Group and aggregate a list of dictionaries by multiple keys

Using pure python, you can do insert into an OrderedDict to retain insertion order:

from collections import OrderedDict

d = OrderedDict()
for l in lst:
    d.setdefault((l['number'], l['favorite']), set()).add(l['color'])

[{'number': k[0], 'favorite': k[1], 'color': v.pop() if len(v) == 1 else v} 
    for k, v in d.items()]   
# [{'color': {'green', 'red'}, 'favorite': False, 'number': 1},
#  {'color': 'red', 'favorite': True, 'number': 1},
#  {'color': 'red', 'favorite': False, 'number': 2}]

---

This can also be done quite easily using the pandas GroupBy API:

import pandas as pd

d = (pd.DataFrame(lst)
       .groupby(['number', 'favorite'])
       .color
       .agg(set)
       .reset_index()
       .to_dict('r'))
d
# [{'color': {'green', 'red'}, 'favorite': False, 'number': 1},
#  {'color': {'red'}, 'favorite': True, 'number': 1},
#  {'color': {'red'}, 'favorite': False, 'number': 2}]

If the condition of a string for a single element is required, you can use

[{'color': (lambda v: v.pop() if len(v) == 1 else v)(d_.pop('color')), **d_} 
     for d_ in d]
# [{'color': {'green', 'red'}, 'favorite': False, 'number': 1},
#  {'color': 'red', 'favorite': True, 'number': 1},
#  {'color': 'red', 'favorite': False, 'number': 2}]

A solution in pure Python would be to use a defaultdict with a composite key. You could use that to merge your values. Afterwards you can create a list again out of that dictionary.

from collections import defaultdict

dct = defaultdict([])

for entry in lst:
    dct[(entry['number'], entry['favorite'])].append(entry['color'])

lst = [{'number': key[0], 'favorite': key[1], color: value if len(value) > 1 else value[0]}
    for key, value in dct.items()]