Questions tagged [rust-polars]
271 questions
0
votes
0 answers
Row-wise scan through a polars LazyFrame
I have a TSV file that is larger than my memory. I want to use the polars crate in the Rust programming language to iterate over this TSV file and benefit from polars' capabilities such as parsing.
Is this possible with the LazyFrame API?
Manuel
- 6,461
- 7
- 40
- 54
0
votes
0 answers
Rust read parquet and iterate over it
How do you efficiently take a parquet file in Rust and iterate over it as a list of structs?
E.g.
struct Reading {
datetime: chrono::DateTime,
value: f64,
}
let filename = "readings.parquet";
let readings: Vec =…
Test
- 962
- 9
- 26
0
votes
0 answers
Parquet into polars from S3 in Rust: error with CloudOptions
I'm completely new to Rust (but experienced coder in general) and I'm struggling with the following problem:
I want to load a Parquet file stored on a Digital Ocean S3 bucket into a polars DataFrame object. (Ideally at a later point I will switch…
gxmw
- 53
- 6
0
votes
1 answer
Rolling Product function in Python Polars
I am looking for rolling_product similar to rolling_sum in polars. Rolling_apply sum is slow compared to rolling_sum.
If it’s not possible, need help to create rolling_prod function. Unable to locate rolling_sum in GIT. Could someone share the…
0
votes
0 answers
Is it possible to read a parquet file from a public S3 bucket using R-polars?
I am testing polars implementation in R and I want to read and process (lazily) a parquet file from a public AWS S3 bucket.
After loading
library(aws.s3)
library(arrow)
library(polars)
I tried to open a connection using arrow::s3_bucket(url) where…
ekleins
- 11
- 3
0
votes
1 answer
Convert &str to f64 using a Rust Polars custom function
My problem can probably be described as being very new to both Rust and Polars. Go easy on me. :)
I'm trying to establish a pattern using custom functions, based on this documentation:…
NotABot83
- 7
- 1
0
votes
1 answer
How to check if element value in Rust-Polars column is 0
I am trying to account for a zero in the denominator when calculating a normalized percentage deviation from the term in the denominator.
Here is the code snippet -
let out: DataFrame = input_df
.lazy()
.with_columns(vec
0
votes
1 answer
Create a Series from A Vec or Vec in rust polars
If I have a a a collection of NaiveDateTimes in a Vec how can I create a ChunkedArray and ultimately a Series from this ?
For ints and floats I can do
let sensor_id_col_values = (0..=20).flat_map(|i| vec![i; 5]).collect();
let sensor_id_col =…
Glenn Pierce
- 720
- 1
- 6
- 18
0
votes
1 answer
Rust polars: Filter a lazy frame by datetime range
I am trying to filter a LazyFrame using a chrono::NaiveDateTime range.
Here is where I currently am at:
use polars::prelude::*;
use polars_lazy::prelude::*;
pub fn keep_range_lazy(
df: &mut DataFrame,
start: &NaiveDateTime,
end:…
Pierre
- 3
- 1
0
votes
1 answer
How to accumulate the rows in a column using polars::prelude::fold_exprs?
In this handbook is an example how to utilise the function polars::prelude::fold_exprs to accumulate the row values over multiple columns horizontally.
let df = df!(
"a" => &[1, 2, 3],
"b" => &[10, 20, 30],
)?;
let out = df
.lazy()
…
LenC
- 69
- 1
- 10
0
votes
0 answers
Polars rust equivalent of pl.struct()
I would like to collect several columns in a dataframe into a struct, but do this in Rust. What is the equivalent of the Python polars.struct() in Rust?
I am new to Rust, rust polars documentation is very difficult for me.
vitcra
- 1
0
votes
1 answer
How do you create a column of a vector/list type in Polars using Rust
I am trying to create a dataframe in Polars that has a column of fixed size vectors that represent a positions in 2d space. Unfortunately the following does not compile.
let df2 = df!(
"a" => &[1, 2, 3],
"b" => &[[1.0, 2.0], [3.0, 4.0],…
Johan
- 660
- 1
- 6
- 13
0
votes
0 answers
How to greedily read string into last column of polars data frame in rust?
I have a csv file with malformed data in the last column. Meaning that the last column can contain " or also , characters.
I would now like to read this csv into a dataframe using polars rust.
I managed to do this using this:
let mut schema =…
Jakob Widauer
- 13
- 3
0
votes
0 answers
Jaccard similarity in Polars Rust
Trying to recreate Jaccard similarity from Python in Rust Polars.
The function compares two lists and returns a float.
Python method
def jacc_sim(x, y):
i_c = len(set.intersection(*[set(x), set(y)]))
u_c = len(set.union(*[set(x), set(y)]))
…
fvg
- 153
- 3
- 9
0
votes
1 answer
Polars add column with counted items from series of list[str]
After splitting a string in multiple 'words', I want to add a new column with the amount of counted items .alias("count").
let df = df! [
"keys" => ["a ab", "a ab abc", "b ba abc abcd", "b ba bbc abcd bbcd"],
"groups" => ["A", "A",…
fvg
- 153
- 3
- 9