If your XSD is not valid, then you cannot be confident that a schema processor will interpret it as you intended. After all, when you feed it gibberish, how is it supposed to know what you mean?
If there's only a little gibberish then it may turn out that your chosen processor will successfully validate some or even all inputs as you wanted, but that's a matter of luck. Other schema processors might validate differently -- even different versions of the same processor might do.
If you are struggling with that, then you should ask yourself why your application bothers with validating XML against XSD in the first place. If you didn't validate, then what kinds of variation from the expected XML form could your application tolerate? Can you be certain that every future version will be the same in that regard?