Questions tagged [rust-polars]
271 questions
1
vote
1 answer
How can I achieve functionality similar to pandas.reindex(new_index, method="ffill") with a datetime column in polars?
In Pandas I can add new rows by their index and forward fill in values without filling any other nulls in the DataFrame:
import numpy as np
import pandas as pd
df = pd.DataFrame(data={"a": [1.0, 2.0, np.nan, 3.0]}, index=pd.date_range("2020",…
Are Haartveit
- 33
- 3
1
vote
1 answer
Polars (Rust), formatting a duration to a string
How could I format a Duration in a HH:MM:SS format?
As a test sample, I have
fn main() {
let df = df! {
"a" => ["2022-11-21T12:00:00"],
"b" => ["2022-11-21T14:00:00"]
}
.unwrap()
.lazy()
.with_column(
…
ohe
- 3,461
- 3
- 26
- 50
1
vote
1 answer
Rust Polars from AWS S3?
Polars guide shows example of loading a file from S3. Unfotunately though, it uses python library pyarrow and a function from_arrow which also seems to be python specific.
I wonder if it would be possible to do the same in pure Rust? Or is my best…
Anatoly Bugakov
- 772
- 1
- 7
- 18
1
vote
1 answer
Polars - How to sort dates and assigning a rank for UniqueID - Python
Relatively new to Polars. I am trying to GroupBy a UniqueID in a Dataframe, and rank based on a datetime column, with the oldest date being ranked as 1 etc...
I saw the code in Pandas but rank() has not been applied as an expression just yet in…
McNickSisto
- 11
- 2
1
vote
0 answers
unable to cast refs returned from `ChunkedArray::chunks` to concrete arrow type
I'm trying to extract the raw buffers from ChunkedArray. The arrow2 documentation suggests doing this by casting a &dyn arrow2::array::Array as its concrete type, see here.
This seems to work fine when I create an arrow buffer directly, however,…
ExpandingMan
- 36
- 2
1
vote
1 answer
How do you vertically concatenate two Polars data frames in Rust?
According to the Polars documentation, in Python, you can vertically concatenate two data frames using the procedure shown in the below code snippet:
df_v1 = pl.DataFrame(
{
"a": [1],
"b": [3],
}
)
df_v2 = pl.DataFrame(
…
Darnock
- 69
- 3
1
vote
1 answer
Rust Polars: How to get the row count of a DataFrame?
I want to filter a Polars DataFrame and then get the number of rows.
What I'm doing now seems to work but feels so wrong:
let item_count = item_df
.lazy()
.filter(not(col("status").is_in(lit(filter))))
.collect()?
…
Lars Francke
- 716
- 7
- 18
1
vote
1 answer
py-polars: groupby_dynamic but via expressions. (timestamp based window functions)
df = pl.DataFrame({
'txn_id': ['0x5...60', '0x1...6d', '0x9...84', '0xc...25', '0x5...50', '0xe...14', '0x2...f3', '0xe...75', '0x3...95', '0x4...4e'],
'txn_grouping': ['0x4...dd', '0x4...dd', '0xf...e2', '0x4...17', '0xe...8b', '0x6...4e',…
Cory Grinstead
- 511
- 3
- 16
1
vote
1 answer
How to sum multi columns by regex in Polars?
I have multi columns which name startswith "ts" like "ts_1, ts_2, ts_3,etc" , I want to sum these f64 value row by row, but I don't know exactly the column names. If I use regex like pl.col('^ts.*$'). How to sum these value?
Hakase
- 211
- 1
- 12
1
vote
0 answers
Custom Expression in Polars
Is it possible to create custom expressions? I understand there is a path for custom functions using the apply or map methods, but would it be possible to create a custom expression in Rust that may then be available on the python side?
dvreed77
- 2,217
- 2
- 27
- 42
1
vote
1 answer
In Python polars convert a json string column to dict for filtering
Hi have a dataframe where I have a column called tags which is a json string.
I want to filter this dataframe on the tags column so it only contains rows where a certain tag key is present or where a tag has a particular value.
I guess I could do a…
Glenn Pierce
- 720
- 1
- 6
- 18
1
vote
1 answer
Load data frame from struct
I'm learning polars rust version and I have a question: is there a way to create a dataframe (or lazy dataframe) by using a struct?
I have some financial data from a data provider that send me a json through http request.
I deserialise this json…
Sigi
- 53
- 8
1
vote
0 answers
Testing if two f64 series are equal
I am trying to write tests for various algorithms in my crate. However, I have come across a bit of a hurdle regarding how f64 values behave. They both return the same value, but I'm not too sure of the precision of the calculated series, because…
Kival M
- 182
- 1
- 10
1
vote
0 answers
Convert arrow2 chunks to Polars Series
I used arrow2 (specifically, io-odbc) to interact with a database. I saved the data as parquet with datatype Vec>>>. Example code below
pub fn write_batch(
path: &str,
schema: Schema,
columns:…
katrocitus
- 45
- 1
- 6
1
vote
2 answers
How to use with_column method to create a calculated column in Polars Rust?
I was trying to create a new computed column based on existing column in polars rust DataFrame. There is a pyspark like with_column method available for that. But in the api documentation there is no example. Here is a example dataframe:
use…
DataPsycho
- 958
- 1
- 8
- 28