Stack Overflow
Stack Overflow

Questions tagged [rayon]

A Rust data-parallelism library that makes it easy to convert sequential computations into parallel Rayon is lightweight and convenient for introducing parallelism into existing code. It guarantees data-race free executions and takes advantage of parallelism when sensible, based on work-load at runtime.

See the official documentation for more information.

123 questions
0
votes
2 answers

Parallel Recursion Fix

Quite new to Rust and trying to tackle toy problems. Trying to write a directory traversal with only Rayon. struct Node { path: PathBuf, files: Vec, hashes: Vec, folders: Vec>, } impl Node { pub fn…
Quin Darcy
  • 1
  • 1
0
votes
1 answer

How to use dynamic struct reference within multithread process in Rust?

I am trying to simulate and transform data with multithreading, while leaving to user what kind of transformation to use, as long as it implements my Transformation trait. I understand broadly why complier doesn't like to use raw Transformation in…
karc
  • 3
  • 1
0
votes
1 answer

Rust returnig iterator with trait bound on its items

I am trying to write a common interface for different types of matrices that provides a way to mutably iterate their rows and modify them. I have the following matrix types: struct NdArrayMatrix { matrix: Array2, } struct ByteMatrix<'a> { …
stomfaig
  • 15
  • 4
0
votes
0 answers

How is the thread pool managed when using Rayon and PyO3?

I am currently trying to speedup a python scientific code with the help of PyO3. Until now, numba's @jit has been doing wonders, but some routines are impossible to parallelize. I decided to turn to Rust and Rayon which provide the tools to…
Quettle
  • 23
  • 4
0
votes
0 answers

How to integrate async data collection with threadpool data processing in Rust

I'd like to improve the integration of my async data collection with my rayon data processing by overlapping the retrieval and the processing. Currently, I pull lots of pages from a web site using normal async code. Once that is complete, I do the…
ahenshaw
  • 1
0
votes
1 answer

Rayon support for itertools tuple_combinations

I'm trying to use Rayon's par_iter over a TupleCombinations struct from itertools as in the code below. use itertools::Itertools; use rayon::prelude::*; fn main() { let v: Vec = vec!(1, 2, 3); let t = v.iter().tuple_combinations::<(_,…
Michael Mior
  • 28,107
  • 9
  • 89
  • 113
0
votes
0 answers

Writing a const generic parallel zip using rayon

I wish to create an IndexedParallelIterator from an array [I; N] where I: IndexedParallelIterator. Let's call it ParConstZip. As I understand, there's a couple of steps here: Create the corresponding non-parallel…
MSR
  • 2,731
  • 1
  • 14
  • 24
0
votes
1 answer

Parallelising file processing using rayon

I've got 7 CSV files (55 MB each) in my local source folder which I want to convert into JSON format and store into local folder. My OS is MacOS (Quad-Core Intel i5). Basically, it is a simple Rust program which is run from a console…
fade2black
  • 546
  • 1
  • 10
  • 26
0
votes
1 answer

Rust claims DashMap into_par_iter trait bounds not satisfied

For some time, my Rust program used a DashMap with a parallel iterator bridge. This was a major performance bottleneck, and I have just found that these days, DashMap supposedly supports direct parallel iteration. So I updated my dependency crates…
Anaphory
  • 6,045
  • 4
  • 37
  • 68
0
votes
1 answer

Error with Rayon, cannot borrow `*self` as mutable, as it is a captured variable in a `Fn` closure

First of all, I'm new toRust. For a game I'm working on I need to call function to change pixels on a vector that is then transformed into an iamge. For that, I have a struct that contains all the context of the game, include said image and…
Miguel Guerrero
  • 11
  • 1
0
votes
1 answer

Use thread-specific pre-allocated data to run tasks in parallel

I want to run some tasks in parallel. Each task uses heap-allocated data. To speed things up, I would like that each thread re-use the same data instead of un-allocating it and re-allocating it just after. Is it feasible? Here is a basic example of…
LucasJavaudin
  • 1
0
votes
2 answers

How to iterate over an array with rayon?

I am new to rust, I am trying to get responses from an array of URLs using rayon use rayon::prelude::*; use reqwest::*; fn main() -> Result<()> { let urls = vec!["https://example.com"]; urls.par_iter().for_each(|url: &&str| -> Result<()>…
Nane
  • 2,370
  • 6
  • 34
  • 74
0
votes
0 answers

Use Rayon to chunk a hash map

Is it possible to use Rayon to chunk the data in a HashMap? I see several chunking methods, but they seem to want to work on a slice (or something similar). use rayon::prelude::*; use std::collections::HashMap; use log::info; fn main() { let foo…
Troy Daniels
  • 3,270
  • 2
  • 25
  • 57
0
votes
0 answers

Multiple Rayon `spawn` with parallel iterators inside

I'm trying to speed up CPU-heavy computations in an async environment (tokio). My initial solution was to use rayon::spawn with a parallel iterator within to actually perform. A minimal example with pseudo-code: for data in vec![data1, data2, data3]…
Daniel Nunes
  • 67
  • 1
  • 8
0
votes
0 answers

Order depended cache for rayon parallel iterators without mutex

I have a computation over the thousands of items that needs to process a sequence of computation where each n iteration need to process data of [n - 1, n, n + 1]. Better to understand this on the chart All the computations inside iterator are same…
Dmitriy Kovalenko
  • 3,437
  • 3
  • 19
  • 39
1 2 3
…
8 9