So i have been playing around with polars
and smartcore
but i have some trouble with converting from a Dataframe
into a DenseMatrix
.
In my code I'm reading in some simple csv data consisting of several columns containing numeric
data.
This gets read into a Dataframe
(polars) but a DenseMatrix
(smartcore) wants as input (for instance) a Vec<Vec<f64>>
.
So i do the conversion but it seems very troublesome, and non ideomatic.
// select some columns and concat the data into a single column containg lists of the column data.
// | Data |
// | [1, 2, 3] |
// | [1, 2, 3] |
// | [1, 2, 3] |
// | [1, 2, 3] |
let n = [concat_list([col("Pclass"), col("SibSp"), col("Parch")])?.alias("data")];
let df = df.clone().lazy().select(n).collect()?;
// iterate through the column and convert each number in the nested list into an f64
let data = df
.column("data")?
.list()?
.into_no_null_iter()
.map(|n| {
n.i64()?
.into_no_null_iter()
.map(|n| n as f64)
.collect::<Vec<f64>>()
})
.collect::<Vec<Vec<f64>>>();
let matrix = DenseMatrix::from_2d_vec(&data)
This solution above works, but it doesn't feel ideomatic, and the documentation for polars is outdated and lacking when it comes to rust. Does anyone have a better solution to this type of problem?