Simply put: generic variance in .NET 4 doesn't support variance for type arguments which are value types.
The reason it can work for reference types is that once the CLR has decided it knows the generic conversion is safe, it can treat all reference values the same way - they have the same representation internally. No actual conversion is required to turn a string
reference into an object
reference, or vice versa if you know it's definitely a reference to a string - it's the same bits, basically. So the generated native code can just treat the references as references, happy in the knowledge that the rules around variance have guaranteed that nothing nasty will happen at the type safety level, and not performing any conversions on the values themselves.
That isn't true for value types, or for reference types where the conversion isn't a "reference conversion" (i.e. a representation-preserving one). That's why you can't write IEnumerable<XName> names = new List<string>();
by the way...