How to use series.isin with different sets for different values?

The problem is that isin expect a sequence of values, and not a Series of sequences. Said differently it allows vectorization on keys but not on values.

So you have to use a non vectorized way here, for example:

df[df.apply(lambda x: x['column2'] in dict1[x['column1']], axis=1)]

You could do with a list comprehension and pandas.concat. In the comprehension, use boolean indexing with logical AND (&) operator:

df_new = pd.concat([df[df['column1'].eq(k) & df['column2'].isin(v)] for k, v in dict1.items()])

[out]

  column1  column2
1       b        2
2       c        6

---

Another approach would be to restructure your dict as a DataFrame and merge:

df_dict = pd.DataFrame([(k, i) for k, v in dict1.items() for i in v], columns=['column1', 'column2'])

df.merge(df_dict, how='inner', on=['column1', 'column2'])