These three classes of invalid polygons would have to be checked for independently.
Case B:
Check to make sure there are no duplicate vertices in your polygon.
Case C:
Check to make sure no vertices land on any edges. Assuming you're using floating point numbers this could get tricky because the floating point numbers would have to evaluate to exactly equal. This can be circumvented by saying they can not be within some EPS
of each other, but that could invalidate some other polygons that are only almost invalid. I would personally use the EPS
myself unless I really needed utmost precision (at which point I don't know what I would do).
Case D:
As you said, this can be found by checking if any edges intersect.
Case E:
Two edges overlap (intersect at infinitely many points) and their vertices don't.
I'm not sure if this needs to be a separate case, but this situation might not be caught by the check for case D (I think you end up dividing by zero). Therefore you would also have to check that the two edges don't correspond to exactly the same line.
I can't think of any other cases for the moment