This isn't a one-size-fit-all case - Therefore we have to look at it case by case.
There are a lot of variables that we must keep in mind - User's internet connection speed, image size, computer capabilities, etc.
I found a few questions on SO that are somewhat relevant to this question. I will include them as I see it beneficial. I am NOT claiming to be absolutely correct.
Performance Argument
AFAIK, browsers cache images the same whether they're in a DIV or an
IMG. In any case, I think this one of those cases where specific
performance is defined as an implementation detail internal to each
rendering engine (and possibly the browsers built around them). As
such, it's both out of our control as designers/developers and subject
to change from browser to browser and version to version. In other
words, I wouldn't spend too much time worrying about it.
Context
Technical differences yes, most notably you can set the width/height
of an IMG tag and it will stretch the image, which can be useful in
some situations.
The main thing you've got to keep in mind though is the context of the
image within the HTML document. If the image is content, say an image
in a gallery, I'd use a IMG tag. If it is just part of the interface I
might use a div instead.
- By Paul
And the response to that is just as important.
That said, you bring up an excellent point about the semantic
difference: an IMG is usually the better choice when the image is a
material part of the page's content, while a CSS technique is usually
preferred when the image is just decorative or for formatting.
This is not performance related directly - More about semantics and accessibility. By - Mr. W.
Then one more for Performance OFF of SO which I feel is directly related to your question.
The version with background images actually gave me a “Document
Complete” after .0225 seconds – while the fully loaded page load time
was roughly the same as the inline image version. Could using all
background images speed up firing of $.document(ready)? It turns out
background-images are only downloaded after the element (containing
div, span etc) is available. This prevents blocking from all the round
trips required to get images.
results: inline image test page

results: background image test page
