Filtering with polars in Rust - Eagerly

Question

I'm trying to do a simple filter with polars in rust :

let mask = df.column("AISLE_ID").unwrap().eq(lit(1));
let filtered_df = df.filter(&mask).unwrap();

But it's not working at all : expected &ChunkedArray<...>, found &bool

I can do it with lazy way but I don't want to clone dataframe

let dfe = df.clone();
    let filtered_df = dfe.lazy().filter(
        col("AISLE_ID").eq(lit(1))
    )
    .collect();

Can you help me ?

You're implicitly cloning in the first example because `df.filter` returns a brand new DataFrame. — BallpointBen, May 28 '23 at 20:15
I need to clone it because if don't do it, I have an error "doesn't have trait copy". A way to do it without cloning ? — Developppeur, May 29 '23 at 07:19
can you post your import ? "use ..." . or else, I guess your code use 'dfe' after filtered_df with filter. — ACE Fly, May 29 '23 at 09:15

score 1 · Accepted Answer · answered May 31 '23 at 14:15

As was mentioned by others, whenever you do a filter, whether it's lazy or not, a copy of the data is performed as a new DataFrame is created. The difference is when the copy is performed (along with optimizations if multiple transformations happen in the scope of the lazy DataFrame).

In your original lazy example, the initial let def = df.copy() is not necessary. The following code compiles and works as expected:

use polars::prelude::*;

fn main() {
    let s0 = Series::new("AISLE_ID", [0, 1, 2].as_ref());
    let s1 = Series::new("temp", [22.1, 19.9, 7.].as_ref());
    let df = DataFrame::new(vec![s0, s1]).unwrap();
    let filtered_df = df.lazy().filter(
        col("AISLE_ID").eq(lit(1))
    )
    .collect();

    println!("{:?}", filtered_df)
}

Returning:

Ok(shape: (1, 2)
┌──────────┬──────┐
│ AISLE_ID ┆ temp │
│ ---      ┆ ---  │
│ i32      ┆ f64  │
╞══════════╪══════╡
│ 1        ┆ 19.9 │
└──────────┴──────┘)

Cargo.toml:

[dependencies]
polars = { version = "0.29.0", features = ["lazy"] }

Filtering with polars in Rust - Eagerly

1 Answers1