Plotrix calls the default barplot parameters and to change the fonts there, you need to use par. For MacOS, it should be as described in this post
Not very good with fonts, hope this is correct:
par(mfrow=c(1,3))
par(family ="sans")
multhist(list(a,b),main="sans")
quartzFonts(palatino = c("Palatino Regular","Palatino Italic",
"Palatino Bold","Palatino Bold Italic"))
par(family ="Palatino")
multhist(list(a,b),main="Palatino")
quartzFonts(avenir = c("Avenir Book", "Avenir Black",
"Avenir Book Oblique", "Avenir Black Oblique"))
par(family ="avenir")
multhist(list(a,b),main="avenir")

So you need to open a new device and do:
quartzFonts(palatino = c("Palatino Regular","Palatino Italic",
"Palatino Bold","Palatino Bold Italic"))
par(family ="Palatino")
multhist(list(a,b),main="Palatino")