How do I iterate over a pyo3 PyObject in Rust?

Question

I have a pre-imported module that I'm calling a method with python gil, something like the following.

Python::with_gil(|py| {
    let res = module.call_method1(py, "my_method", (arg1, arg2))?;
})

This returns the rust object PyObject, however what this returns is a python list. I want to iterate over this list to convert the internals into something I can use in Rust (it's a python list of Numpy arrays, I'm using the numpy/ndarray crates).

I'm a little confused as to how I'm meant to iterate over this. If I try cast_as to a PyList, I get the warning: UnsafeCell<PyObject> cannot be shared between threads safely. It seems extract does not work either.

How do I iterate over this PyObject? Thanks.

Edit: Adding further details as requested

The returned value from python is a List[numpy.ndarray] if you are using the python Typing system. As the lengths of each numpy array could be different, I cannot just convert it all into a numpy array in python and pass it through. An example output is below:

[array([214.17725372, 192.78236675, 354.27965546, 389.84558392,
          0.99999297])]

What I've tried in Rust:

let pylist = res.cast_as::<PyList>(py)?;

Fails to compile with: UnsafeCell<PyObject> cannot be shared between threads safely.
let pylist = res.extract::<PyList>(py)?;

Fails to compile with: the trait 'PyClass' is not implemented for 'PyList'. Please note I have use pyo3::prelude::*; at the top.
let pyany = res.extract::<Vec<PyArray1<f64>>>(py)?;

Fails to compile with: the trait bound 'Vec<PyArray<f64, Dim<[usize; 1]>>>: pyo3::FromPyObject<'_>' is not satisfied. This PyArray is from the numpy crate.

Please share the code for what you've tried. Please include a minimal example including your python code. — PitaJ, Feb 23 '22 at 18:13

score 0 · Answer 1 · answered Oct 29 '22 at 18:44

I see that you returning a list of numpy arrays with dtype=float. If you are open to using another dependency, there is rust-numpy which lets you map numpy arrays to ndarrays in Rust.

ndarrays have a .to_vec() method to convert to standard Rust containers.

In your case you could do the following (this is not tested):

use numpy::PyArray1;
use pyo3::{types::IntoPyDict, PyResult, Python};

Python::with_gil(|py| {
    let res: Vec<&PyArray1<f32>> = module // <-- Notice the return type
                                 .call_method1(py, "my_method", (arg1, arg2))?
                                 .extract()?; // <--  Notice the extract
})

Mappings between Python and Rust types can be found here. You can then use the res to do further computations.

println!("I just got a numpy array from python {:?}", res[0]);
println!("I converted that to a vector here: {:?}", res[0].to_vec());

How do I iterate over a pyo3 PyObject in Rust?

1 Answers1