Turn generic method into trait object safe method

Question

I would like to make an adapter that removes generic parameters (to produce a trait object) as in the example below.

use std::ops::Deref;

fn make_dyn_box<I, S>(iter_in: I)
where
    I: Iterator<Item = S>,
    S: Deref<Target = u8>,
{
    let mut iter_out = iter_in.map(
        |s| -> Box<dyn Deref<Target = u8>> {Box::new(s)}
    );
    take_dyn_box(&mut iter_out)
}

fn take_dyn_box<'a: 'b, 'b>(
    iter: &'a mut (dyn 'a + Iterator<Item = Box<dyn 'b + Deref<Target = u8>>>),
) { }

Is there a way to accomplish this without a heap allocation, using only safe code, and no external dependencies?

The below is an idea of what I want, but the borrow checker doesn't allow this.

use std::ops::Deref;

fn make_dyn<I, S>(iter_in: I)
where
    I: Iterator<Item = S>,
    S: Deref<Target = u8>,
{
    let mut item = None;
    let item = &mut item;
    let mut iter_out = iter_in.map(|s| -> &dyn Deref<Target = u8> {
        item.replace(s);
        Option::as_ref(item).unwrap()
    });
    take_dyn(&mut iter_out)
}

fn take_dyn<'a: 'b, 'b>(
    iter: &'a mut (dyn 'a + Iterator<Item = &'b (dyn 'b + Deref<Target = u8>)>),
) { }

When you say "adapter" do you mean specifically an iterator adapter? — James Gilles, Jul 13 '20 at 00:46
If you can take an iterator of references, this [becomes very easy](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=38a87fb22af0afedc8cd794d7ed535b1). However, if you have an iterator that owns the values it iterates over (or perhaps more accurately, yields owned values), you might have to create a custom iterator that somehow stores at least each value yielded, but yields the trait object (as a reference). — EvilTak, Jul 13 '20 at 01:20

James Gilles · Accepted Answer · 2020-08-21T16:54:02.917

One simple way to do it is to require that the input iterator return references. This compiles:

fn make_dyn<'b, I, S>(iter_in: I)
where
    I: Iterator<Item = &'b S>,
    S: Deref<Target = u8> + 'b,
{
    let mut iter_out = iter_in.map(|s| -> &dyn Deref<Target = u8> {
        s as _
    });
    
    take_dyn(&mut iter_out)
}

fn take_dyn<'a: 'b, 'b>(
    _iter: &'a mut (dyn 'a + Iterator<Item = &'b (dyn 'b + Deref<Target = u8>)>),
) { }

Note that it's good style to have iterator adapters return iterators that can be further manipulated:

fn make_dyn_<'b, I, S>(iter_in: I) -> impl Iterator<Item = &'b (dyn 'b + Deref<Target = u8>)>
where
    I: Iterator<Item = &'b S>,
    S: Deref<Target = u8> + 'b,
{
    iter_in.map(|s| -> &dyn Deref<Target = u8> {
        s as _
    })
}

(You could also define this as a generic struct that implements the Iterator trait.)

Now: if you don't want to require that the input iterator return references, then there's no way to return your new iterator.

What you're doing in the example code is creating a little buffer within your iterator, and returning references to it.

If that buffer is stored in your iterator struct, what you're trying to create is called a streaming iterator and can't be implemented currently. This extremely long blog post explains why; essentially, it would require large and hairy extensions to Rust's type system.

You can do something like this, if you rearrange your code to have users pass closures into your functions. Then you control when the closures get called, so you can store the returned values into a buffer and pass references to the buffer into the closure. But that's not as ergonomic as the usual Iterator interface.

This is sort of what you've done with your example code... I'm not exactly sure why it doesn't work. If you changed take_dyn to take a single &'b (dyn 'b + Deref<Target = u8>)> and called it repeatedly it should work though.

Took some time to fight through that blog post, but thanks for the background info; it was quite helpful. — dontarius, Jul 23 '20 at 00:15

Turn generic method into trait object safe method

1 Answers1