I was researching this very point and came across this question but didn't feel it had been answered fully. However, I did find this handy article - Design Guidelines Update: Factories vs. Constructors - by Krzysztof Cwalina (a a Principal Architect on the .NET Framework). It is worth reading the entire article, but here is a brief summary of the main points:
The most common and consistent way to create an instance of a type is
via its constructor. However, sometimes a preferable alternative is to
use the Factory pattern.
Do prefer constructors over Factories as they are generally more
consistent and convenient than specialized construction mechanisms.
Do implement Factory operations as methods, not properties.
Do return instances as method return values, not as out parameters.
Consider using a Factory if you need more control over the creation
patterns of instances.
Consider naming Factory methods by concatenating “Create” and the name
of the type being created.
Consider naming Factory types by concatenating the name of type being
created and “Factory.”
Do not implement your Factory using a static method if the
construction operation must be available for subclasses to specialize.
Do use a Factory for conversion style operations.
Do use a Factory if an operation requires parameter information which
feels unnatural to pass to a constructor.
Do not use a Factory in situations where the creation operation will
be used as a core scenario for instantiation of a type. In these
situations, discoverability and consistency are paramount.
Consider using a constructor instead of a Factory. Only after this
thought process should you proceed with the implementation of a
Factory.
Factories are also often handy to enable type specialization through
polymorphism.
Do use a Factory if API users will be coding to a base class or
interface whose implementation is subject to change over time.
Do use Factory methods in cases where a developer might not know which
type to construct, for instance when coding against a base type or
interface.
Do implement Factory operations as virtual instance methods rather
than static if they must support polymorphic extension.
Do use the Singleton pattern for Factories which are instance based so
as not to force developers to instantiate a Factory type just to
invoke one of its members.
Do use a Factory method if a constructor would be insufficient to
describe the operation being performed, and the additional information
gained from an individually named Factory makes an operation’s purpose
clearer.