You've got some pixels down at about 305 (which I guess is blue) so most of your pixels are over 320 and so all in the brown/red end of your palette.
What you might want is a palette that changes over the peak in your data. You can do this by creating a palette that starts with a bunch of colours for the low values. For example:
> rep(rev(rainbow(n=5)),c(5,1,1,1,1))
[1] "#CC00FFFF" "#CC00FFFF" "#CC00FFFF" "#CC00FFFF" "#CC00FFFF" "#0066FFFF"
[7] "#00FF66FF" "#CCFF00FF" "#FF0000FF"
That has 9 colours, and the first five are identical. In this case, 5/9 of the range of the data would all be that colour, and the highest 4/9 of the dataset would be coloured by the full range of the data. Since your peak spans about that amount, you'll get the effect you are after. Adjust the palette to suit.
Note that you should always ask what question any graphic is trying to answer, and adjust colours/scales etc to help answer that question, rather than just trying to make it not look "dull", unless this is for the wall of a gallery.