There is no need to use either of them for most characters. Just enter the characters, in UTF-8, or in ISO-8859-1, whichever you are using. If you don’t know how to type a no-break space, for example, and don’t want to learn it, the entity reference
and the character reference  
are equivalent, so use whichever you find less confusing. (The latter is not an entity reference, so “Entity Number” is a misnomer.)
Only for “<” and for “&”, when used as data characters, will you need “escape” notations, and for them, I cannot figure out any reason why you would use character references, instead of the reasonably mnemonic entity references <
and &
.