I'm trying out Galleria, with the Classic theme.
With a single tweak (changing the classic theme to always show the captions, which does not affect this), my test code works, but the first image shown does not have a caption. Subsequent images do.
Demo : http://www.ski-review.com/ski_news/galleria/
Firebug reveals that the relevant div class (galleria-info-title) has display:none set on page load, but the subsequent divs for the following images do not.
I've played around with various options which either have no affect, or make it worse (no caption at all). It seems odd that an out of the box plugin and theme would have sucn an inconsistent behaviour.