Why does Option<&String> not coerce to Option<&str>?

Question

I have an Option<String> and a function that takes an Option<&str>. I thought Option::as_ref would work, because usually &String converts automatically to &str, but not here. I get this error:

sandbox.rs:6:27: 6:37 error: mismatched types:
 expected `core::option::Option<&str>`,
    found `core::option::Option<&collections::string::String>`
(expected str,
    found struct `collections::string::String`) [E0308]

While this answer describes how to convert from one to the other, I'd still like to know why the &String isn't "coerced" (if that's the right term) into &str like it usually is.

Answer 1

Your expectation is fair. If the compiler has magic to transform &String to &str, it should be able to also transform Option<&String> to Option<&str> or (for that matter) AnyType<&String> to AnyType<&str>.

But the compiler in this case has very little magic. Paradoxically coercions emerged from an attempt to reduce the magic in the compiler^(*). To understand this, you will have to understand the link between coercions and reborrows and follow me through a little excursus in Rust history.

A while back, in Rust you could relatively often see a construct like &*x: a "reborrow". For pointer types like Box, you wanted to be able to dereference them with * to get to the value inside. And if you needed a &i32 but had a x that was a Box<i32>, you could reborrow it with &*x, which is really the composition of two operations, dereference the Box with * and take a new reference to its content with &. Box needed a lot of compiler magic to enable this.

So the thinking went: if we allowed anyone to decide what the * dereference does for their types, we would reduce the magic needed for custom pointers like Box, Rc, Arc, allowing new ones to be defined in libraries... And thus Deref was born.

But then, the Rust devs went one step further to reduce the unsightly reborrows. If you were passing a &Box<i32> to something expecting a &i32 you might have needed to do this (which still compiles by the way):

fn main() {
    let a = &Box::new(2);
    test(&**a); // one * to dereference the Box, 
                // one * to dereference the &
                // reborrow with &
}

fn test(a: &i32) { println!("{}", *a) }

So (the good Rust devs went) why don't we make reborrowing automated and let people just write test(a)?

When there is a type T (eg Box<i32>) that derefs to U (i32), we'll let &T (&Box<i32>) "coerce" to &U (&i32).

This you may recognize as the current coercion rule. But all the magic the compiler performs is to try reborrowing for you by sprinkling *s (calling deref) as needed. More of a little parlor trick, actually.

Now, back to the AnyType<&String> to AnyType<&str> coercion. As we've seen, the compiler is much less magic than we were led to believe and dereferencing-reborrowing &*AnyType<&String> does not lead to the expected result. Even if you managed to implement Deref for AnyType and got *AnyType<&String> to deref to AnyType<str>, reborrowing the result would still yield &AnyType<str>, not AnyType<&str>.

So Rust's mechanism for coercion can't be used for this. You need to explicitly tell Rust how to take the &String out of AnyType and put a &str back in.

As one workaround for your specific scenario, if your function taking a Option<&str> is only one and under your control, you could generalize it to take a Option<T> where T:AsRef<str> instead, like this:

fn test<T: AsRef<str>>(o: Option<T>) {
    if let Some(s) = o { println!("{}", s.as_ref()) }
}

_{(*) the Rust devs are decidedly anti-magic, those muggles!}

Answer 2

There are two coersions in common use in Rust: auto-ref and auto-deref.

Auto-ref can only convert T to &T or &mut T, so isn't relevant here.

Auto-deref automatically calls Deref::deref. Option<T> doesn't implement Deref, so this cannot hold. String, however, does deref to &str, which is why you can see that coercion.

We know Option doesn't implement Deref since Deref::deref returns a pointer with the same lifetime as &self, and the lifetime requirements on pointers means that the pointer must thus live as long as &self, so with few exceptions the pointer must be to some pre-existing object rather than one created in the deref call.

This doesn't mean that the above is the only reason it doesn't implement Deref, although it is a sufficient one.

Answer 3

The Rust language has defined a way to implicitly convert a &T into a &U, for specific pairs of T and U, through the Deref trait. It's important to understand that in some cases, &U is a constructed value. For example, neither String nor &String contain a &str, so the deref() method needs to create the &str manually.

However, the language has not defined a way to implicitly convert a X<&T> into a X<&U>. For that to work, the compiler would have to know a way to initialize a X<&U> given a X<&T> (and it's going to be different for different values of X). You can't simply take the binary representation of a X<&T> and reinterpret that as a X<&U>. In fact, an Option<&String> and an Option<&str> don't even have the same size! And even if the sizes matched, the pointers would usually not be the same.

In order to develop a generic solution to that problem, Rust would have to implement higher-kinded types (not available as of Rust 1.6). There could also be an ad hoc solution for that specific problem, but it would become redundant when HKT are implemented.