Rust equivalent of concurrent code written in Go?

Question

I'm stuck at doing multithread feature in Rust. I'm trying to do translate my code written in Go that updates a map's value while iterating and make a new thread. (simplified codes)

my_map := make(map[string]string)
var wg sync.WaitGroup
wg.Add(len(my_map ))

for key := range my_map {
    go func(key string) {
        defer wg.Done()
        stdout, _ := exec.Command(key, "some command").Output()

        lock.Lock()
        defer lock.Unlock()
        my_map[key] = "updating map value while iterating"  // eg stdout
    }(key)
}

I tried so far like this

let mut my_map = HashMap::new();

...

for (key, value) in my_map.iter_mut() {
    // Should update map value, tried with crossbeam_utils:thread;

    thread::scope(|s| {
        s.spawn(|_| {
            let cmd = format!("some command {key}");  // will borrow key
            let cmd: Vec<&str> = cmd.as_str().split(" ").map(|s| s).collect();

            let proc = Popen::create(
                &cmd,
                PopenConfig {
                    stdout: Redirection::Pipe,
                    stdin: Redirection::None,
                    stderr: Redirection::None,
                    ..Default::default()
                },
            );

            let mut proc = proc.expect("Failed to create process.");

            let (out, _) = proc.communicate(None).unwrap();
            let stdout = out.unwrap();

            *value = "I will update value with something new";
        });
    })
    .unwrap();
}

@ChayimFriedman `thread::scope` joins all the threads inside, which means there isn't actually any multithreading in the Rust code; it's just spawning a thread then waiting for it to finish every iteration. — Aplet123, Aug 23 '22 at 12:08
I'll just throw in here that Rust threads are heavy OS threads that come with their own stack unlike Go's green threading model. You might want to look into an [asynchronous Rust runtime](https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html#popular-async-runtimes) instead. — isaactfa, Aug 23 '22 at 12:31
I'm trying to understand what your Go code does, but it seems to me that your using variables that you never declared (`found_ssids` and `ssid`); and not using those that you declare (`my_map` and `key`). Also, the `.map(|s| s)` is a bit weird. — jthulhu, Aug 23 '22 at 12:41
@BlackBeans sorry for the simplified code, I changed names to post it and didnt know it wasn't changed! But in there as Rust, it updates key value while iterating with spawning a thread — alfex4936, Aug 24 '22 at 00:47
@Masklinn will it be possible to iterate all keys of map and borrow those keys? I'm not sure if I can do like Go, while iterating every key, in background(?) they're updating values (long task) concurrently and join after the loop — alfex4936, Aug 24 '22 at 02:13
@alfex4936 that is the entire point of thread::scope: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c0d6298a2ddb0223e3ec278337866b08 — Masklinn, Aug 24 '22 at 06:32

score 0 · Answer 1 · answered Aug 23 '22 at 12:21

I create a Vector of handlers which would be similar to a wait group, and I push each thread there. I then iterate over each handle and .unwrap() it to make sure it finished.

    let mut handles: Vec<std::thread::JoinHandle<()>> = Vec::new();

    for (key, value) in my_map.iter_mut() {
        handles.push(thread::spawn(move || {
            let cmd = format!("some command");
            let cmd: Vec<&str> = cmd.as_str().split(" ").map(|s| s).collect();

            let proc = Popen::create(
                &cmd,
                PopenConfig {
                    stdout: Redirection::Pipe,
                    stdin: Redirection::None,
                    stderr: Redirection::None,
                    ..Default::default()
                },
            );

            let mut proc = match proc {
                Ok(proc) => proc,
                Err(e) => {
                    panic!("Failed to create process: {:?}", e)
                }
            };

            let (out, _) = proc.communicate(None).unwrap();
            let stdout = out.unwrap();

            *value = "I will update value with something new";
        }));
   }
    for handle in handles {
        handle.join().unwrap(); // This won't block other threads, but it will make sure it can unwrap every thread before it continues. In other words, this only runs as long as your longest running thread.
    }

Thanks! but how can I use it if I borrow a key in the thread (for example, key is some command) and print out everything in map after joining? Felt like I need to use Arc for my map but I'm stuck at here — alfex4936, Aug 24 '22 at 01:24

Rust equivalent of concurrent code written in Go?

1 Answers1