Stack Overflow
Stack Overflow

Questions tagged [python-polars]

Polars is a DataFrame library/in-memory query engine.

The Polars core library is written in Rust and uses Arrow, the native arrow2 Rust implementation, as its foundation. It offers Python and JavaScript bindings, which serve as a wrapper for functionality implemented in the core library.

Links

1331 questions
0
votes
1 answer

How to assign multiple values to specific locations in a series in an expr?

In pandas, one can use logical indexing to assign items: s = pd.Series(['a', 'b', 'c', 'd', 'e']) idx = [True, False, True, False, True] s[idx] = ['x', 'y', 'z'] In polars, we can do this with set_at_idx: s = pl.Series(['a', 'b', 'c', 'd',…
NedDasty
  • 192
  • 1
  • 8
0
votes
1 answer

python-polars create new column by dividing by two existing columns

in pandas the following creates a new column in dataframe by dividing by two existing columns. How do I do this in polars? Bonus if done in the fastest way using polars.LazyFrame df = pd.DataFrame({"col1":[10,20,30,40,50],…
tommyt
  • 309
  • 5
  • 15
0
votes
2 answers

How to swap column values on conditions in python polars?

I have a data frame as below: df_n = pl.from_pandas(pd.DataFrame({'last_name':[np.nan,'mallesh','bhavik'], 'first_name':['a','b','c'], …
myamulla_ciencia
  • 1,282
  • 1
  • 8
  • 30
0
votes
1 answer

How to map a dict of expressions to a dataframe

Problem Is there some nice/efficient/best way how to get a dict of polars expressions being applied (and evaluated) on a dataframe (given a column value for match and same+other column values as a part of the expression evaluation)? Setup import…
Michal Chromčák
  • 1
  • 3
0
votes
1 answer

Match the behavior of prefix_sep argument to pandas.get_dummies, in polars

I have a variable driver_age and some levels 16_to_25, 25_to_34, etc. I would like the dummy encoded columns to have names like driver_age@16_to_25. I have the following workaround, but it is incompatible with LazyFrames. prefix_sep = "@" for col in…
johnhopfensperger
  • 13
  • 3
0
votes
0 answers

polars getting 'PyDataFrame is not defined' error when reading from csv

trying to read csv into polars dataframe but getting the following error 'PyDataFrame' is not defined. I installed polars latest version for python 3.8 #basic script to read from csv import polars as pl data =…
Ross
  • 99
  • 8
0
votes
1 answer

Randomly drop % of rows by condition in polars

Imagine we have the following polars dataframe: Feature 1 Feature 2 Labels 100 25 1 150 18 0 200 15 0 230 28 0 120 12 1 130 34 1 150 23 1 180 25 0 Now using polars we want to drop every row with Labels == 0 with 50%…
Janikas
  • 418
  • 1
  • 8
0
votes
4 answers

Combine different values of multiple columns into one column

Need help to "translate" a python example to rust. The python example was given here Here is the code snippet I try to make work: use polars::prelude::*; fn main() { let s1 = Series::new("Fruit", &["Apple", "Apple", "Pear"]); let s2 =…
Robert
  • 131
  • 1
  • 7
0
votes
1 answer

How to convert rows into columns given an arbitrary step size in polars (in a single step)?

I've managed to solve this problem in two steps. import polars as pl text = "a brown fox jumps over a lazy dog's head" step = 3 df = pl.DataFrame({"a":text.split(" ")}) first = df.filter(pl.arange(0, pl.count())%step==0) second =…
pedrosaurio
  • 4,708
  • 11
  • 39
  • 53
0
votes
3 answers

How to compare date values from rows in python polars?

I have a dataframe with date of births as pl.DataFrame({'idx':[1,2,3,4,5,6], 'date_of_birth':['03/06/1990','3/06/1990','11/12/2000','01/02/2021','1/02/2021','3/06/1990']}) Here I would like to compare date of birth(Format:…
myamulla_ciencia
  • 1,282
  • 1
  • 8
  • 30
0
votes
2 answers

Can I get elements from column of lists by list of indexes?

In (Py)Polars there is method of subset list elements in column of lists according to list of indexes in other column? I.e. arr.get() accepts only Integer and not accept Expressions (like pl.col('prices').arr.get(pl.col('idxs').arr.first())) ? Can I…
Some lazy dude
  • 3
  • 3
0
votes
0 answers

ERROR: Could not build wheels for polars, which is required to install pyproject.toml-based projects

When I install polars in the new environment: note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for polars Failed to build polars ERROR: Could not build wheels for polars, which is…
ztsweet
  • 1
  • 2
0
votes
1 answer

How to ignore NULL fields while concatenating strings from multiple columns in python polars?

I have a dataframe with person names with these fields - last, first and middle names, i'm trying to concatenating these fields to get a full_name column in a dataframe as below. dfl.with_columns( pl.concat_str([pl.col('last_name'), …
myamulla_ciencia
  • 1,282
  • 1
  • 8
  • 30
0
votes
0 answers

Polars: How do I best clip a numerical column to a certain quantile?

I would like to be able to clip numerical values in a DataFrame based on the result of an expression on that DataFrame. However, the clip function only accepts floats or ints, not expr. Given the following: df = pl.DataFrame({'x': [0,…
Bart Helder
  • 13
  • 3
0
votes
1 answer

Repeat rows in a Polars DataFrame based on column value

I would like to expand the following Polars dataframe by repeating rows based on values in the quantity column. Original DataFrame: Fruit Quantity Apple 2 Banana 3 Expected…
NFern
  • 1,706
  • 17
  • 18
1 2 3
…
88 89