How to write trait & impl with lifetimes for iterators?

Question

I'm trying to understand how to write a trait and an impl for it for my own types that will process some input data. I'm starting with a simple example where I want to process the input 1, 2, 3, 4 with a trait Processor. One implementation will skip the first element and double all remaining inputs. It should therefore look like this:

trait Processor {} // TBD

struct SkipOneTimesTwo;
impl Processor for SkipOneTimesTwo {} // TBD

let numbers = vec![1, 2, 3, 4];
let it = numbers.iter();
let it = Box::new(it);

let proc = SkipOneTimesTwo;
let four_to_eight = proc.process(it);
assert_eq!(Some(4), four_to_eight.next());
assert_eq!(Some(6), four_to_eight.next());
assert_eq!(Some(8), four_to_eight.next());
assert_eq!(None, four_to_eight.next());

So my assumption is that my trait and the corresponding implementation would look like this:

trait Processor {
    // Arbitrarily convert from `i32` to `u32`
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>>;
}

struct SkipOneTimesTwo;
impl Processor for SkipOneTimesTwo {
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>> {
        let p = it.skip(1).map(|i| 2 * (i as u32));
        Box::new(p)
    }
}

This code doesn't work as-is. I get the following error:

7 |     let four_to_eight = proc.process(it);
  |                                      ^^ expected `i32`, found reference
  |
  = note:   expected type `i32`
          found reference `&{integer}`
  = note: required for the cast to the object type `dyn Iterator<Item = i32>`

If my input data were very large, I wouldn't want the entire dataset to be kept in-memory (the whole point of using Iterator), so I assume that using Iterator<T> should stream data through from the original source of input until it is eventually aggregated or otherwise handled. I don't know what this means, however, in terms of what lifetimes I need to annotate here.

Eventually, my Processor may hold some intermediate data from the input (eg, for a running average calculation), so I will probably have to specify a lifetime on my struct.

Working with some of the compiler errors, I've tried adding 'a, 'static, and '_ lifetimes to my dyn Iterator<...>, but I can't quite figure out how to pass along an input iterator and modify the values lazily.

Is this even a reasonable approach? I could probably store the input Iterator<Item = i32> in my struct and impl Iterator<Item = u32> for SkipOneTimesTwo, but then I would presumably lose some of the abstraction of being able to pass around the Processor trait.

score 1 · Accepted Answer · edited Jan 18 '21 at 20:54

1

All iterators in Rust are lazy. Also, you don't need to use lifetimes, just use into_iter() instead of iter() and your code compiles:

trait Processor {
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>>;
}

struct SkipOneTimesTwo;

impl Processor for SkipOneTimesTwo {
    fn process(&self, it: Box<dyn Iterator<Item = i32>>) -> Box<dyn Iterator<Item = u32>> {
        let p = it.skip(1).map(|i| 2 * (i as u32));
        Box::new(p)
    }
}

fn main() {
    let numbers = vec![1, 2, 3, 4];
    let it = numbers.into_iter(); // only change here
    let it = Box::new(it);
    
    let pro = SkipOneTimesTwo;
    let mut four_to_eight = pro.process(it);
    assert_eq!(Some(4), four_to_eight.next());
    assert_eq!(Some(6), four_to_eight.next());
    assert_eq!(Some(8), four_to_eight.next());
    assert_eq!(None, four_to_eight.next());
}

playground

edited Jan 18 '21 at 20:54

vallentin

23,478
6
59
81

answered Jan 18 '21 at 20:51

pretzelhammer

13,874
15
47
98

This certainly works for what I will need to do, so thank you! I was also curious about the ability to use `.iter` and thus not consume the input. Does that use case differ significantly in the implementation vs this one? – user655321 Jan 18 '21 at 21:02
1

@user655321 the implementation would almost be identical, instead of calling `.into_iter()` you would call `.iter().copied()` where [`.copied()`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.copied) lazily copies the elements and leaves the original input unconsumed. – pretzelhammer Jan 18 '21 at 21:06
Surely you're joking. There's little that's difficult about iterators if you're passing around values, not references. This is not helpful. – gnkdl_gansklgna Jul 13 '21 at 17:25

How to write trait & impl with lifetimes for iterators?

1 Answers1