Stack Overflow
Stack Overflow

Questions tagged [python-polars]

Polars is a DataFrame library/in-memory query engine.

The Polars core library is written in Rust and uses Arrow, the native arrow2 Rust implementation, as its foundation. It offers Python and JavaScript bindings, which serve as a wrapper for functionality implemented in the core library.

Links

1331 questions
0
votes
1 answer

How do you cast to an unsigned int in polars?

I tried pl.col('foo').cast(np.uint32) and I got a NotImplementedError. I ask because col.str.lengths() returns a column of type UInt32 and columns need to be the same type for joins.
user2966505
  • 67
  • 4
0
votes
1 answer

Searching a DataFrame in polars

I'm trying to write a small python script which reads a .parquet file with the following schema: a b c d 0 x 2 y 2 1 x z The script takes the following arguments: one input file multiple columns multiple search strings (can be…
FrozenPie
  • 3
  • 1
  • 2
0
votes
2 answers

create a polars dataframe containing unique values from a set of CSVs

I have +3000 CSVs with +10 columns. What I need is to get all unique values from just two of these. I am able to read unique values in polars: import polars as pl df1 = pl.read_csv("test1.biobank.tsv.gz", sep='\t', dtype={"#chrom": pl.Utf8},…
darked89
  • 332
  • 1
  • 2
  • 17
0
votes
1 answer

Get column as pl.Series not as pl.Dataframe in polars

I'm trying to get the column of a Dataframe as Series. df['a'] returns allways a pl.Dataframe. Right now I'm doing it this way pl.Series('GID_1',df['GID_1'].to_numpy().flatten().tolist()) I don't think that's the best way to do it. Does anyone…
seb2704
  • 390
  • 1
  • 5
  • 17
0
votes
1 answer

Polars python equivalent to glimpse and summary in R

I couldn't find a function that would summarize the content in the polars dataframe just like glimpse and summary do it in R?
aajkaltak
  • 1,437
  • 4
  • 20
  • 28
0
votes
0 answers

Py-Polars DateTime Conversion

I am currently exploring Py-Polars and are having some difficulties with getting the Date32 format in its dataframe. I have tried the following means: Conversion from Pandas to PyPolars directly import pandas as pd import pypolars as pyp a =…
Sphere
  • 9
  • 3
-1
votes
1 answer

Convert python list to polars dataframe

I have a list and a variable, eg: myvar = 'KEY_1' mylist = ['apple', 'banana', 'peach'] I'd like to convert them to a Polars dataframe of something like: PROD PARAM1 PARAM2 PARAM3 KEY_1 apple banana peach How can I do that?
lmocsi
  • 550
  • 2
  • 17
-1
votes
0 answers

pl.sql_expr() giving incorrect result while performing simple calculation like "2+4/5"

I am using polars sql_expr() for evaluating the dataframe and while evaluating the result found that sql_expr is giving incorrect output for a simple calculation like "2+4/5". for example: polars…
dhruv bothra
  • 25
  • 4
-1
votes
1 answer

Combining multiple groups in Polars

I have a dataframe like this: category year count apple 2022 5 apple 2021 8 banana 2022 1 cold 2022 9 cold 2021 2 warm 2022 1 warm 2021 3 I need to group the rows based on a pre-set list of groupings ('fruit', 'temperature')…
zoned post meridiem
  • 113
  • 4
-1
votes
1 answer

Why am I getting ComputeError while performing left join between two Polars Dataframes?

The below code snippet is inside a function and the function is inside a class. I am using polars instead of pandas. I tried running the function and it showed me an error while performing the left join. import polars as pl inventory =…
Abhiram M V
  • 3
  • 5
-1
votes
1 answer

Is there a way to reduce resource usage when reading and writing large dataframes with polars?

For my specific problem I have been converting ".csv" files to ".parquet" files. The CSV files on disk are about 10-20 GB each. Awhile back I have been using ".SAS7BDAT" files of similar size to convert to ".parquet" files of similar data but now I…
Adeib Bashar
  • 9
  • 1
-1
votes
1 answer

How to debug large Polars pl.select - narrow down offending row

when I encounter a panic error in a select query, how do I narrow down which is / are the offending expressions Edit: simply by looking at the logs / setting a Param. Without having to alter the code into a sequential binary search for…
user1441053
  • 680
  • 1
  • 5
  • 10
-1
votes
2 answers

Python-Polars update DataFrame function similar to Pandas DataFrame.update()

Thanks for the prompt responses. Based on the responses, I have modified the question and also provided numeric code example. I am from Market Research industry. We analyse survey databases. One of the requirements of the survey tables is that blank…
Suresh Nimbalkar
  • 17
  • 4
-1
votes
1 answer

Can I easily remove quotes from text output generated using Polars (https://www.pola.rs/) in Python?

I have a Python Flask app that recently switched from using Pandas to Polars for some dataframe handling. The pertinent code is shown here: data = { 'Text': ['Virginia Woolf, Mrs. Dalloway', 'College website corpus', 'Presidential inaugural…
SummittDweller
  • 21
  • 3
-1
votes
1 answer

How to assign column values based on another column iteratively with Polars

For these two dfs, I want to check for each i in df1["TS"] if df["TS"] == df1["TS}, then assign the value in "Dr" that corresponds to i to the "mmsi" column: df = pl.DataFrame({"TS": [1, 2, 3, 4, 5, 6, 7], "mmsi":[11,12,13,14,15,16,17]}) df1 =…
kalinka227
  • 25
  • 4
1 2 3
…
88
89