pandas concatenate two df removing duplicates code example
Example 1: remove duplicates based on two columns in dataframe
df.drop_duplicates(['A','B'],keep= 'last')
Example 2: concatenate dataframes pandas without duplicates
>>> df1
A B
0 1 2
1 3 1
>>> df2
A B
0 5 6
1 3 1
>>> pandas.concat([df1,df2]).drop_duplicates().reset_index(drop=True)
A B
0 1 2
1 3 1
2 5 6