Stack Overflow
Stack Overflow

Questions tagged [rust-polars]

271 questions
4
votes
2 answers

Rust Polars: Is it possible to explode a list column into multiple columns?

I have a function which returns a list type column. Hence, one of my columns is a list. I'd like to turn this list column into multiple columns. For example: use polars::prelude::*; use polars::df; fn main() { let s0 = Series::new("a", &[1i64,…
Anatoly Bugakov
  • 772
  • 1
  • 7
  • 18
4
votes
2 answers

How to parse date string with days and months without 0 padding in rust version of polars?

I am reading a csv file with date in month day year format (e.g. "11/15/2022"). But month and day do not have 0 padding. Following is my test code use polars::prelude::*; use polars_lazy::prelude::*; fn main() { let df = df![ "x" =>…
Kushdesh
  • 1,118
  • 10
  • 16
4
votes
2 answers

How to avoid deep copy when using groupby in polars rust?

I have a dataset where I need to do groupby operation on different columns. Here is minimal working code using polars version "0.21.1" use polars::prelude::*; use polars_lazy::prelude::*; use polars::df; fn main(){ let df = df![ "x1" => ["a",…
Kushdesh
  • 1,118
  • 10
  • 16
4
votes
2 answers

Efficiently build a Polars DataFrame row by row in Rust

I would like to create a large Polars DataFrame using Rust, building it up row by row using data scraped from web pages. What is an efficient way to do this? It looks like the DataFrame should be created from a Vec of Series rather than adding rows…
Steven Murdoch
  • 151
  • 1
  • 5
4
votes
1 answer

Joining dataframes using rust polars in Python

I am experimenting with polars and would like to understand why using polars is slower than using pandas on a particular example: import pandas as pd import polars as pl n=10_000_000 df1 = pd.DataFrame(range(n), columns=['a']) df2 =…
SultanOrazbayev
  • 14,900
  • 3
  • 16
  • 46
4
votes
0 answers

Rust polars dataframe has not attribute "to_ndarray"

Here we go again with another problem I can't solve, although it sound so easy. I am following Rust polars documentation about dataframes: https://docs.rs/polars/0.14.2/polars/frame/struct.DataFrame.html I am trying to simply implement the easies…
FrankNrg92
  • 71
  • 4
3
votes
2 answers

& operator in predicate (filter) - How to base the filter on multiple column values?

I am trying to filter a Polars dataframe based on two columns values. In the the rust user guide I could only find a predicate based filter based on one column value. How do I add additional columns in the predicate and return a boolean? I am trying…
jeevt
  • 81
  • 5
3
votes
1 answer

How to write df as parquet to s3 in polars rust?

I am a bit out of ideas as after doing this let mut cursor = Cursor::new(Vec::new()); let parquet_bytes = ParquetWriter::new(cursor) .with_statistics(true) .with_compression(ParquetCompression::Snappy) .finish(df) .unwrap(); I no…
andy8203
  • 51
  • 2
3
votes
1 answer

Dataframe conversion from pandas to polars -- difference in the final dimensions

I'm trying to convert a Pandas Dataframe to a Polar one. I simply used the function result_polars = pl.from_pandas(result). Conversion proceeds well, but when I check the shape of the two dataframe I get that the Polars one has half the size of the…
081N
  • 35
  • 4
3
votes
2 answers

Polars DataFrame save to sql

Is there a way to save Polars DataFrame into a database, MS SQL for example? ConnectorX library doesn’t seem to have that option.
Dennis L
  • 33
  • 3
3
votes
0 answers

Why does polars finish method of SerWriter trait require mutable reference

In the definition of polars SerWriter trait: pub trait SerWriter where W: Write, { fn new(writer: W) -> Self; fn finish(&mut self, df: &mut DataFrame) -> Result<(), PolarsError>; } The finish method require a mut reference. Since the…
Bartek
  • 155
  • 1
  • 4
3
votes
0 answers

Polars join on array items without explode/groupby

a follow up from Polars lazyframe - add fields from other lazyframe as struct without a `collect`. I now want to join on array items. Currently the only way i know of doing this would be to first explode the array, perform the join, do a groupby,…
Cory Grinstead
  • 511
  • 3
  • 16
3
votes
1 answer

Polars lazyframe - add fields from other lazyframe as struct without a `collect`

I am trying to populate a new field containing a struct of all of the other fields from another lazyframe based on a predicate. While the examples are in python, I am open to answers in python or rust. companies = pl.DataFrame({ "id": [1], …
Cory Grinstead
  • 511
  • 3
  • 16
3
votes
1 answer

How can I add a column of empty arrays to polars.DataFrame?

I am trying to add a column of empty lists to a polars dataframe in python. My code import polars as pl a = pl.DataFrame({'a': [1, 2, 3]}) a.with_columns([pl.lit([]).alias('b')]) throws Traceback (most recent call last): File "", line 1,…
Dimitrius
  • 564
  • 6
  • 21
3
votes
1 answer

Is there a Pandas Profiling like implemention built on polars?

We use Pandas and Pandas Profiling extensively in our projects to generate profile reports. We were going to explore using Polars as a Pandas alternative and wanted to check if there were any implementations like Pandas Profiling built on top of…
Shashi Deshetti
  • 1,354
  • 2
  • 11
  • 22
1
2
3
…
18 19