1

I am trying to read a file into a string messages defined on line #14. The file contains several blocks where each block starts with a number. After I read the file contents into the string messahes, each block is separated by newline and each line in a block is separated by __SEP__. I would like to use par_split() on the string messages, process each block using rayon and collect output from each block into a vector vec_finale.g. by calling collect() on line 54 or some similar mechanism to produce a vector that contains vec_local on line 53 produced by each block. Any pointers on how I can achieve this are highly appreciated.

My code is as follows:

fn starts_with_digit_or_at_sign(inp: &str) -> bool {
    let mut at_sign_found = false;
    if inp.len() > 0 {
        let ch = inp.chars().next().unwrap();
        if ch.is_numeric() || ch == '@' {
            return true;
        }
    }
    return false;
}
fn main() {
    let filepath = "inp.log";
    let data = std::fs::read_to_string(filepath).expect("file not found!");
    let mut messages: String = String::from("");
    let separator_char = '\n';
    let separator: String = String::from("__SEP__");
    let mut found_first_message = false;
    let mut start_of_new_msg = false;
    let mut line_num = 0;
    for line in data.lines() {
        line_num += 1;
        if line.len() > 0 {
            if starts_with_digit_or_at_sign(line) {
                start_of_new_msg = true;
                if !found_first_message {
                    found_first_message = true;
                } else {
                    messages.push(separator_char);
                }
            }
            if found_first_message {
                if !start_of_new_msg {
                    messages.push_str(&separator);
                }
                messages.push_str(line);
                if start_of_new_msg {
                    start_of_new_msg = false;
                    let mut tmp = String::from("Lnumber ");
                    tmp.push_str(&line_num.to_string());
                    messages.push_str(&separator);
                    messages.push_str(&tmp);
                }
            }
        }
    }
    messages.par_split(separator_char).for_each(|l| {
        println!(
            "line: '{}' len: {}, {}",
            l,
            l.len(),
            rayon::current_num_threads()
        );
        let vec_local: Vec<i32> = vec![l.len() as i32];
    }); // <-- line 54
}

Output produced by the cide is as follows:

line: '1__SEP__Lnumber 1__SEP__a__SEP__b__SEP__c' len: 41, 8
line: '3__SEP__Lnumber 9__SEP__g__SEP__h__SEP__i' len: 41, 8
line: '2__SEP__Lnumber 5__SEP__d__SEP__e__SEP__f' len: 41, 8
line: '4__SEP__Lnumber 13__SEP__j__SEP__k__SEP__l' len: 42, 8

File inp.log is as follows:

1
a
b
c
2
d
e
f
3
g
h
i
4
j
k
l
BallpointBen
  • 9,406
  • 1
  • 32
  • 62
user1653651
  • 31
  • 2

1 Answers1

0

I was able to resolve the issue by using par_lines() instead as follows:

    let tmp: Vec<_> = messages.par_lines().map(|l| proc_len(l)).collect();
...
...
...

fn proc_len(inp: &str) -> Vec<usize> {
    let vec: Vec<usize> = vec![inp.len()];
    return vec;
}
user1653651
  • 31
  • 2