Interesting. Looks like this issue is related to what is chosen as the base line to align the text labels. This could be seen clearly when switching to geom_label
where we see that for the clipped label the base line chosen for the alignment is not the end of the "p". Hence the "p"s get clipped off:
ggplot(df) +
geom_point(mapping = aes(x = x, y = y)) +
annotate("label", x = mean(df$x), y = -Inf, label = "Clipped",
hjust = 0.5, vjust = "inward", size = 12, colour = "red", label.padding = unit(0, "lines")) +
annotate("label", x = mean(df$x), y = Inf, label = "Not Clipped",
hjust = 0.5, vjust = "inward", size = 12, colour = "blue", label.padding = unit(0, "lines"))

One possible fix would be to switch to ggtext::GeomRichtext
:
library(ggplot2)
library(ggtext)
ggplot(df) +
geom_point(mapping = aes(x = x, y = y)) +
annotate(ggtext::GeomRichtext, x = mean(df$x), y = -Inf, label = "Clipped",
hjust = 0.5, vjust = "inward", size = 12, colour = "red",
label.size = 0, fill = NA, label.padding = unit(0, "lines")) +
annotate(ggtext::GeomRichtext, x = mean(df$x), y = Inf, label = "Not Clipped",
hjust = 0.5, vjust = "inward", size = 12, colour = "blue",
label.size = 0, fill = NA, label.padding = unit(0, "lines"))
