How do I return a Result containing every error from an iterator of Results, not just the first one?

Question

I'm trying to implement a simple interpreter in Rust, for which I have created a Tokens struct, which takes source characters and produces either a Token or a ScanError inside a Result:

pub struct Tokens<'src> {
    chars: Chars<'src>,
}

impl<'src> Iterator for Tokens<'src> {
    type Item = Result<Token, ScanError>;

    fn next(&mut self) -> Option<Result<Token, ScanError>> {
        //  ...
    }
}

Since Result implements FromIterator, it is simple to collect the result to either the first ScanError or a vector of Tokens:

fn scan_tokens(source: &str) -> Result<Vec<Token>, ScanError> {
    let iter = Tokens {
        chars: source.chars(),
    };

    iter.collect()
}

In the case of multiple errors I really want to return every error:

fn scan_tokens(source: &str) -> Result<Vec<Token>, Vec<ScanError>> {
    // what goes here?
}

It isn't possible as far as I know to implement my own version of FromIterator because neither that trait or Result are local to my crate. Can anyone suggest a clean way of doing this?

I have written an implementation using partition on the iterator, then unwrapping each Result, below, but it's not fun to read and doesn't feel like good use of iterators:

type T = Vec<Result<Token, ScanError>>;
fn scan_tokens(source: &str) -> Result<Vec<Token>, Vec<ScanError>> {
    let iter = Tokens {
        chars: source.chars(),
    };

    let (tokens_results, error_results): (T, T) = iter.partition(|result| result.is_ok());
    let errors: Vec<ScanError> = error_results
        .into_iter()
        .map(|result| result.unwrap_err())
        .collect();

    if errors.len() > 0 {
        return Err(errors);
    }

    Ok(tokens_results
        .into_iter()
        .map(|result| result.unwrap())
        .collect())
}

Answer 1

unwrapping each Result

I would use itertools' partition_map to avoid the need to unwrap:

use itertools::{Either, Itertools}; // 0.8.0

fn iterator() -> impl Iterator<Item = Result<i32, bool>> {
    vec![Ok(1), Err(false), Ok(2), Err(true), Ok(3)].into_iter()
}

fn example() -> Result<Vec<i32>, Vec<bool>> {
    let (values, errors): (Vec<_>, Vec<_>) = iterator().partition_map(|v| match v {
        Ok(v) => Either::Left(v),
        Err(e) => Either::Right(e),
    });

    if errors.is_empty() {
        Ok(values)
    } else {
        Err(errors)
    }
}

See also:

• What's the most idiomatic way of working with an Iterator of Results?
• How do I stop iteration and return an error when Iterator::map returns a Result::Err?
• How do I perform iterator computations over iterators of Results without collecting to a temporary vector?

You could also use the fact that Option and Result implement IntoIterator to avoid the exact unwrap, although this still processes one collection twice:

fn example2() -> Result<Vec<i32>, Vec<bool>> {
    let (values, errors): (Vec<_>, Vec<_>) = iterator().partition(|result| result.is_ok());

    if errors.is_empty() {
        Ok(values.into_iter().flat_map(Result::ok).collect())
    } else {
        Err(errors.into_iter().flat_map(Result::err).collect())
    }
}

See also:

• Why does `Option` support `IntoIterator`?

Answer 2

An imperative solution is often the most expressive and efficient way to implement some algorithm. It's Rust, not Haskell; not everything needs to be functional.

fn scan_tokens(source: &str) -> Result<Vec<Token>, Vec<ScanError>> {
    let iter = Tokens {
        chars: source.chars(),
    };
    let mut tokens = Vec::new();
    let mut errors = Vec::new();
    for result in iter {
        match result {
            Ok(token) => {
                tokens.push(token);
            }
            Err(e) => {
                errors.push(e);
            }
        }
    }
    if errors.is_empty() {
        Ok(tokens)
    } else {
        Err(errors)
    }
}