How to make a color scale with sharp transition in ggplot2

In case you are still interested in a solution for this, you can add guide = guide_colourbar(nbin = <some arbitrarily large number>) to scale_fill_gradientn(). This increases the number of bins used by the colourbar legend, which makes the transition look sharper.

# illustration using nbin = 1000, & weighted colours below the cutoff
plot.cutoff <- function(cutoff){
  p <- ggplot(data = test,
              aes(x = as.factor(x), y = y, fill = log(y))) + 
    geom_col(width = 1) +
    scale_fill_gradientn(colours = c("red4", "red", "yellow", "green"), 
                         values = scales::rescale(log(c(0.01, cutoff - 0.0000000000000001, 
                                                        cutoff, 0.2))), 
                         breaks = c(log(cutoff)), 
                         label = c(cutoff),
                         guide = guide_colourbar(nbin = 1000))
  return(p)
}

cowplot::plot_grid(plot.cutoff(0.05),
                   plot.cutoff(0.06),
                   plot.cutoff(0.08),
                   plot.cutoff(0.1),
                   ncol = 2)

(If you find the above insufficiently sharp at very high resolutions, you can also set raster = FALSE in guide_colourbar(), which turns off interpolation & draws rectangles instead.)