Questions tagged [rust-polars]
271 questions
1
vote
0 answers
Update values in a Rust Polars Dataframe
I have a dataframe which looks like this (with a lot more rows and columns)-
┌────────┬────────────┬────────────┐
│ email ┆ col1 ┆ col2 │
│ --- ┆ --- ┆ --- │
│ str ┆ str ┆ str …
parmesant
- 73
- 4
1
vote
1 answer
unable to convert json to dataframe, polars panicked
use reqwest::blocking::get;
use polars::prelude::*;
use serde::{Deserialize, Serialize};
use std::io::Cursor;
#[derive(Serialize, Deserialize, Debug)]
struct StockZhAHist {
date: String,
open: f64,
close: f64,
high: f64,
low:…
Arthur Zhang
- 107
- 8
1
vote
0 answers
Is there a better way to handle set-like operations with Polars without iterating over with Python functions?
Is there a better way to handle set-like operations with Polars?
A problem: I have a data with connected ids, where I want to group all ids together and attach to every group of ids some unique id, in this simple case, just a positive integer.
This…
Dmitry Russ
- 21
- 2
1
vote
0 answers
Adding new methods to dataframes
I see that you can extend the API in Python with custom namespaces, but there is no such functionality in rust it seems.
However, there are optional add-ons for polars like repeat_by. How do I extend the polars data/lazy-frames with my own functions…
The Unfun Cat
- 29,987
- 31
- 114
- 156
1
vote
0 answers
How to use Rust Polars to convert a string date into a date type (StrpTimeOptions not in scope)
I have been trying many examples to learn to use polars with dates, but every example uses StrpTimeOptions and I can't find any crate or features that bring that into scope. I also cannot find any documentation on this. Some of the examples that…
Roger Bos
- 11
- 1
1
vote
1 answer
Groupby two dataframes at the same time
I have the following code
import polars as pl
df = pl.DataFrame(
{
"grpbyKey": [1, 1, 1, 2, 2, 2],
"val": ["One"] * 3 + ["Two"] * 3
}
)
df2 = pl.DataFrame(
{
"grpbyKey": [1, 1, 2, 2, 2, 3],
"val2": ["One"] * 2 + ["Two"] * 3 +…
The Unfun Cat
- 29,987
- 31
- 114
- 156
1
vote
1 answer
fill_null in LazyFrame equivalent to strategy on DataFrames
I've been trying to use Lazyframes instead of Dataframes more often due to performance reasons. Unfortunately, not all features available in DataFrames are available for LazyFrames, one of these being the .fill_null method, that takes a…
Dinaiscoding
- 992
- 1
- 7
- 16
1
vote
1 answer
Does Polars support UUID?
I have a time series of string-formatted UUIDs, and I would like Polars to translate them into u128 numbers for better storage and querying.
Similar to what we do with dates:
....str.strptime(pl.Datetime, fmt="%Y-%m-%dT%H:%M:%S.%fZ",…
Jeremy Chone
- 3,079
- 1
- 27
- 28
1
vote
1 answer
How to do ewm_mean in rust polars?
In python we can do:
df.with_columns([
pl.col("myCol").ewm_mean(50)
])
But how do we do the same in rust? The following doesn't work:
df.with_columns([
col("myCol").ewm_mean(50)
])
It fails with No method named ewm_mean found in…
Corvus
- 7,548
- 9
- 42
- 68
1
vote
1 answer
Polars show_graph method
I am starting to use Polars as a replacement for Pandas. I am very interested in using the show_graph() method to my team to show some of the benefits. One of the biggest benefits I can see is the use of scan_csv. However I am seeing some oddities…
magladde
- 614
- 5
- 23
1
vote
1 answer
Polars groupby aggregating by sum, is returning a list of all unique values instead of actual sum
I'm trying to do a aggregation from a polars DataFrame. But I'm not getting what I'm expecting.
This is a minimal replication of the issue:
import polars as pl
# Create a DataFrame
df = pl.DataFrame({"category": ["A", "A", "B", "B", "B"],
"value":…
Jose Nuñez
- 11
- 1
- 4
1
vote
1 answer
Create a polars dataframe from postgres sql in a generic way
Hi I am trying to read from postgres into a polars frame in a generic way.
I have read a post here
Rust: Read dataframe in polars from mysql
about reading from mysql and want to change this so I don't need to handle the columns for each new query.
I…
Glenn Pierce
- 720
- 1
- 6
- 18
1
vote
1 answer
Groupby and cut on a Lazy DataFrame in Polars
import numpy as np
import polars as pl
def cut(_df):
_c = _df['x'].cut(bins).with_columns([pl.col('x').cast(pl.Int64)])
final = _df.join(_c, left_on='x', right_on='x')
return final
groups = ["A"]*500 + ["B"]*500
bins = [0, 100, 200,…
BoreBoar
- 2,619
- 4
- 24
- 39
1
vote
1 answer
Serialize Polars `dataframe` to `serde_json::Value`
In Polars, it is simple enough to serialize a DataFrame to a json string: JsonWriter::new(dest).finish(&df)?.
Is it possible to serialize to a json Value — a serde_json::Value — that is the Value::Array of records? Obviously, I could write the json…
BallpointBen
- 9,406
- 1
- 32
- 62
1
vote
1 answer
Logging in Polars
I don't see any sort of logging implemented in Pola-rs.
I am curious to know what desgin decisions/ philosophy went behind choosing not to implement any loggings in pola-rs and how is the library debugged, if anyone is willing to throw some light at…
iamsmkr
- 800
- 2
- 10
- 29