R4DS error comparison (1) is possible only for atomic and list types

You probably had another package with a function (filter) loaded and masking dplyr filter

quick and dirty fix:

dplyr::filter()

instead of

filter()

use library(dplyr)

library(dplyr)

ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
  geom_point(mapping = aes(color = class)) +
  geom_smooth(
    data = filter(mpg, class == "subcompact"),
    se = FALSE)

The filter() function comes from the dplyr package. Be sure you've loaded it before running those lines. Otherwise, you're running a comparison with class(), the built-in function, rather than mpg$class.