Most efficient way to fill a vector from back to front

Question

I am trying to populate a vector with a sequence of values. In order to calculate the first value I need to calculate the second value, which depends on the third value etc etc.

let mut bxs = Vec::with_capacity(n);

for x in info {
    let b = match bxs.last() {
        Some(bx) => union(&bx, &x.bbox),
        None => x.bbox.clone(),
    };
    bxs.push(b);
}
bxs.reverse();

Currently I just fill the vector front to back using v.push(x) and then reverse the vector using v.reverse(). Is there a way to do this in a single pass?

The `unsafe` way would probably be faster, but I am not sure whether it's worth it. — WiSaGaN, May 11 '16 at 06:28
FWIW, I would do it exactly as you've described it, push in-order and then reverse it. Since you are sure to have some profiling to test if a one-pass solution is more efficient, can you show us the results of that profiling that indicates the two-pass solution is not efficient? — Shepmaster, May 11 '16 at 12:39
@Shepmaster I think you are right that two passes are better. The tradeoff between simplicity of the code and performance is not worth a single pass. — Justin Raymond, May 11 '16 at 16:23
Although I'm not sure why two passes would ever be more efficient than one pass. The only reason I could think of is the direction would confuse the pre-fetcher, but Intel CPUs can detect streams of memory accesses in either forward or backward direction (http://stackoverflow.com/questions/1950878/c-for-loop-indexing-is-forward-indexing-faster-in-new-cpus). This cofr is part of a raytracer, where performance is very important and the code segment may run millions of times for a large image. — Justin Raymond, May 11 '16 at 16:28

score 7 · Answer 1 · edited May 11 '16 at 08:24

7

Is there a way to do this in a single pass?

If you don't mind adapting the vector, it's relatively easy.

struct RevVec<T> {
    data: Vec<T>,
}

impl<T> RevVec<T> {
    fn push_front(&mut self, t: T) { self.data.push(t); }
}

impl<T> Index<usize> for RevVec<T> {
    type Output = T;
    fn index(&self, index: usize) -> &T {
        &self.data[self.len() - index - 1]
    }
}

impl<T> IndexMut<usize> for RevVec<T> {
    fn index_mut(&mut self, index: usize) -> &mut T {
        let len = self.len();
        &mut self.data[len - index - 1]
    }
}

edited May 11 '16 at 08:24

oli_obk

28,729
6
82
98

answered May 11 '16 at 06:51

Matthieu M.

287,565
48
449
722

Is the idea that you'd actually leave the data in the backwards order, but make accesses work back-to-front? Clever. You'd have to watch out for any other cases (iterators, etc.) when you expose those. – Shepmaster May 11 '16 at 11:59
@Shepmaster: Yes, it's the idea, and indeed it means reversing all accesses. – Matthieu M. May 11 '16 at 12:33
Is there a crate for this, by chance? That implements all the methods in reverse (iteration etc). – nirvana-msu Jun 16 '22 at 11:57
Well, found [rev_slice](https://github.com/scottmcm/rev_slice/blob/master/src/lib.rs) which has all slice methods implemented, although on a newtype which is not as useful/generic as it could have been. – nirvana-msu Jun 16 '22 at 12:45

score 5 · Accepted Answer · answered May 13 '16 at 19:24

The solution using unsafe is below. The unsafe version is slightly more than 2x as fast as the safe version using reverse(). The idea is to use Vec::with_capacity(usize) to allocate the vector, then use ptr::write(dst: *mut T, src: T) to write the elements into the vector back to front. offset(self, count: isize) -> *const T is used to calculate the offset into the vector.

extern crate time;
use std::fmt::Debug;
use std::ptr;
use time::PreciseTime;

fn scanl<T, F>(u : &Vec<T>, f : F) -> Vec<T>
    where T : Clone,
          F : Fn(&T, &T) -> T {
    let mut v = Vec::with_capacity(u.len());

    for x in u.iter().rev() {
        let b = match v.last() {
            None => (*x).clone(),
            Some(y) => f(x, &y),
        };
        v.push(b);
    }
    v.reverse();
    return v;
}

fn unsafe_scanl<T, F>(u : &Vec<T> , f : F) -> Vec<T>
    where T : Clone + Debug,
          F : Fn(&T, &T) -> T {
    unsafe {
        let mut v : Vec<T> = Vec::with_capacity(u.len());

        let cap = v.capacity();
        let p = v.as_mut_ptr();

        match u.last() {
            None => return v,
            Some(x) => ptr::write(p.offset((u.len()-1) as isize), x.clone()),
        };
        for i in (0..u.len()-1).rev() {
            ptr::write(p.offset(i as isize), f(v.get_unchecked(i+1), u.get_unchecked(i)));
        }
        Vec::set_len(&mut v, cap);
        return v;
    }
}

pub fn bench_scanl() {
    let lo : u64 = 0;
    let hi : u64 = 1000000;
    let v : Vec<u64> = (lo..hi).collect();

    let start = PreciseTime::now();
    let u = scanl(&v, |x, y| x + y);
    let end= PreciseTime::now();
    println!("{:?}\n in {}", u.len(), start.to(end));

    let start2 = PreciseTime::now();
    let u = unsafe_scanl(&v, |x, y| x + y);
    let end2 = PreciseTime::now();
    println!("2){:?}\n in {}", u.len(), start2.to(end2));
}

You [probably don't need to use pointer offsets](https://play.rust-lang.org/?gist=fe502c56f90352547b08a70fb8d5f991&version=stable&backtrace=0). Also, your code has non-idiomatic Rust, like the UFCS call to `Vec::set_len`, explicit types, `&Vec` instead of `&[T]`, wider `unsafe` blocks than needed, `return` statements, spaces before `:` in types, etc. You may want to get idiomatic reviews at some point. — Shepmaster, May 13 '16 at 19:37
Anyone with more experience with the language is welcome to improve the solution. — Justin Raymond, May 13 '16 at 19:42
@JustinRaymond You're missing a few `assert!` / `assert_eq!` calls there. That `unsafe` code doesn't look safe to me. — wizzwizz4, Jun 27 '19 at 08:51

Most efficient way to fill a vector from back to front

2 Answers2

Linked

Related