remove duplicates from dictionary python code example

Example 1: removing duplicates from dictionary python

student_data = {'id1': 
   {'name': ['Sara'], 
    'class': ['V'], 
    'subject_integration': ['english, math, science']
   },
 'id2': 
  {'name': ['David'], 
    'class': ['V'], 
    'subject_integration': ['english, math, science']
   },
 'id3': 
    {'name': ['Sara'], 
    'class': ['V'], 
    'subject_integration': ['english, math, science']
   },
 'id4': 
   {'name': ['Surya'], 
    'class': ['V'], 
    'subject_integration': ['english, math, science']
   },
}

result = {}

for key,value in student_data.items():
    if value not in result.values():
        result[key] = value

print(result)

Example 2: create dictionary without removing duplicates from dataframe

{k: g.to_dict(orient='records') for k, g in df.groupby(level=0)}
# {'bob': [{'age': 20, 'name': 'bob'}, {'age': 30, 'name': 'bob'}],
#  'jim': [{'age': 25, 'name': 'jim'}]}

Example 3: python remove duplicates from list of dict

# set the dict to a tuple for hashability, then use {} for set literal and retrn each item to dict. 
[dict(t) for t in {tuple(d.items()) for d in l}]
# using two maps()
list(map(lambda t: dict(t), set(list(map(lambda d: tuple(d.items()), l)))))