merge two dataframes pandas exclude duplicates code example

Example 1: remove duplicates based on two columns in dataframe

df.drop_duplicates(['A','B'],keep= 'last')

Example 2: concatenate dataframes pandas without duplicates

>>> df1
   A  B
0  1  2
1  3  1
>>> df2
   A  B
0  5  6
1  3  1
>>> pandas.concat([df1,df2]).drop_duplicates().reset_index(drop=True)
   A  B
0  1  2
1  3  1
2  5  6