Stack Overflow
Stack Overflow

Questions tagged [python-polars]

Polars is a DataFrame library/in-memory query engine.

The Polars core library is written in Rust and uses Arrow, the native arrow2 Rust implementation, as its foundation. It offers Python and JavaScript bindings, which serve as a wrapper for functionality implemented in the core library.

Links

1331 questions
4
votes
1 answer

polars native way to convert unix timestamp to date

I'm working with some data frames that contain Unix epochs in ms, and would like to display the entire timestamp series as a date. Unfortunately, the docs did not help me find a polars native way to do this, and I'm reaching out here. Solutions on…
tenxsoydev
  • 370
  • 2
  • 10
4
votes
1 answer

Has anyone used Polars and Seaborn or Matplotlib together?

Has anyone used a Polars dataframe with Seaborn to graph something? I've been working through a notebook on Kaggle that used Pandas, and I wanted to refactor it to Polars. The dataframe I'm working with looks like this: PassengerID…
Jordan
  • 45
  • 5
4
votes
0 answers

Use polars to read many small json files from S3 in parallel

I have seen the following polars documentation: https://pola-rs.github.io/polars-book/user-guide/multiple_files/intro.html#reading-and-processing-in-parallel Is there a way to create a query plan to read many small json files from an S3 bucket? This…
Clay
  • 2,584
  • 1
  • 28
  • 63
4
votes
1 answer

Converting string to DateTime Polars

I have a Polars dataframe with a column of type str with the date and time in format 2020-03-02T13:10:42.550. I want to convert this column to the polars.datetime type. After reading this post Easily convert string column to pl.datetime in Polars, I…
Johnas
  • 296
  • 2
  • 5
  • 15
4
votes
2 answers

Retrieve date from datetime column in polars

Currently when I try to retrieve date from a polars datetime column, I have to write sth. similar to: df = pl.DataFrame({ 'time': [dt.datetime.now()] }) df = df.select([ pl.col("*"), pl.col("time").apply(lambda x:…
Alex
  • 439
  • 5
  • 16
4
votes
1 answer

df.select('some_col').to_numpy() and df['some_col'].to_numpy() converge to a different result

data = {"a": [1, 2], "b": [3, 4]} df = pl.DataFrame(data) print(df['a'].to_numpy() [1 2] print(df.select('a').to_numpy()) [[1][2]] On the one hand it's adwised to not use the df['some_column'] syntax, but on the other hand they yield different…
supersick
  • 261
  • 2
  • 14
4
votes
2 answers

Compare 2 tables in Polars and select a value based on that comparison

I have a table like this in polars: arrival_time Train 08:40:10 112 19:31:26 134 An I have another table that defines the period of the day based on the hours: Time Period 08:00:00 Early 16:00:00 Afternoon What I am…
luis.martinez.pro
  • 41
  • 3
4
votes
2 answers

How to show Polars Dataframe in PyCharm

In PyCharm you have the ability to show a Pandas Dataframe with the SciView tool. Is this also possible with Polars or would I have to spam print statements? (I also opened a PyCharm support ticket)
zacko
  • 179
  • 2
  • 9
4
votes
1 answer

How can I concat polars dataframes that have different columns

In pandas it happens automatically, just by calling pd.concat([df1, df2, df3]) and the frame that didn't have the column previously just gets a column filled with NaNs. In polars I get a 'shape error' with the message that the columns differ (11…
zacko
  • 179
  • 2
  • 9
4
votes
2 answers

Fast apply of a function to Polars Dataframe

What are the fastest ways to apply functions to polars DataFrames - pl.DataFrame or pl.internals.lazy_frame.LazyFrame? This question is piggy-backing off Apply Function to all columns of a Polars-DataFrame I am trying to concat all columns and hash…
Jenobi
  • 368
  • 4
  • 12
4
votes
2 answers

Add column based on groupby

I'm trying to port a pandas script to polars. I have a dataset that looks like…
Nil
  • 2,345
  • 1
  • 26
  • 33
4
votes
1 answer

How to perform computations easily between every column in a polars DataFrame and the mean of that column

Environment macos: monterey node: v18.1.0 nodejs-polars: 0.5.3 Goal Subtract every column in a polars DataFrame with the mean of that column. Pandas solution In pandas the solution is very concise thanks to…
Valley
  • 51
  • 5
4
votes
2 answers

Rust Polars: Is it possible to explode a list column into multiple columns?

I have a function which returns a list type column. Hence, one of my columns is a list. I'd like to turn this list column into multiple columns. For example: use polars::prelude::*; use polars::df; fn main() { let s0 = Series::new("a", &[1i64,…
Anatoly Bugakov
  • 772
  • 1
  • 7
  • 18
4
votes
1 answer

Does Polars support creating a dataframe from a nested dictionary?

I'm trying to create a polars dataframe from a dictionary (mainDict) where one of the values of mainDict is a list of dict objects (nestedDicts). When I try to do this I get an error (see below) that I don't know the meaning of. However, pandas does…
Tiaan
  • 43
  • 1
  • 3
4
votes
2 answers

How to select columns by data type in Polars?

In pandas we have the pandas.DataFrame.select_dtypes method that selects certain columns depending on the dtype. Is there a similar way to do such a thing in Polars?
astrojuanlu
  • 6,744
  • 8
  • 45
  • 105
1 2 3
…
88 89