Stack Overflow
Stack Overflow

Questions tagged [rust-polars]

271 questions
0
votes
3 answers

Repeating a date in polars and exploding it

I have a polars dataframe with two date columns that represent a start and end date and then a value that I want to repeat for all dates in between those two dates so that I can join those on other tables. Example input…
slugmagug
  • 13
  • 5
0
votes
1 answer

How to spilt a big DataFrame into Vec by group in Polars

I stored some financial market data in a Polars DataFrame. As for analysis, it is is fast to run some groupby("date").agg() action. But in a realtime scenario , the new data is coming time by time, I don't want to concat the new data with old data…
Hakase
  • 211
  • 1
  • 12
0
votes
1 answer

How to create a polars dataframe on the basis of previous row

I'm trying to create a polar data frame in python. Dataframe format is: timestamp(secs) Counter 164323232 2 I'm given only the first row. Now I need to create a dummy dataframe (say 100 rows) on the basis of this first row. Each row…
linkedtechs
  • 1
  • 1
0
votes
1 answer

Offsets in downcast_iter and series slicing in Polars

In Polars, I'm seeing a return result different than what I would expect when using slicing with series and trying to get the offsets. I'm creating a Series, then slicing it: // Make a vec of 3 items, called foo, bar baz let string_values: Vec<&str>…
Josh
  • 501
  • 1
  • 7
  • 11
0
votes
1 answer

Upsampling a polars dataframe with groupby

I'm trying to upsample a Polars dataframe while grouping by a particular column. In the following example, I wish to group by 'fruit' and then upsample by…
NFern
  • 1,706
  • 17
  • 18
0
votes
1 answer

Removing last row and extending dataframe prints dataframe with the removed row

Why is it when I try and remove the last row a dataframe and insert a new row, it still shows the previous row I tried to remove with the new row? if self.dataframes.minute()["broker_time"].utf8().unwrap().into_iter().any(|i| i.unwrap() ==…
Mass
  • 1
  • 2
0
votes
1 answer

In polars, nested when-then-otherwise gives unexpected behavior in groupby / window context

I have a custom expression to wrap around the "rank" expression to ignore nulls. def rank(_exp,method='average',reverse=False): #Fill nans so as not to affect ranking fill = -np.Inf if reverse else np.Inf tmp =…
lowmotion
  • 1
  • 1
0
votes
1 answer

How to properly set-up Graphviz for Polars on Mac or Windows?

Polars has a powerful feature called df.show_graph(optimized=True). Have been trying to get this installed on my Windows and Mac computer, system info is below. Windows OS = Windows 10 Architecture = x86_64 MacBook OS = Monterey Arch= arm64 (Apple…
Jenobi
  • 368
  • 4
  • 12
0
votes
1 answer

Rust polars : unexpected befaviour of when().then().otherwise() in groupby-agg context

I have a complicated mapping logic which I seek to execute within groupby context. The code compiles and doesn't panic, but results are incorrect. I know the logic implementation is correct. Hence, I wonder if when-then-otherwise is supposed to be…
Anatoly Bugakov
  • 772
  • 1
  • 7
  • 18
0
votes
1 answer

How to get first n chars from a str column/Utf8Chunked in rust polars

What's the alternative of pandas : data['ColumnA'].str[:2] in rust polars? My first guess was: let x = Utf8Chunked::new("ColumnA", &["Pear", "apple", "toly", "x"]); let y = x.slice(0, 2); I'd like to get an array/ChunkedArray/Utf8Chunked which…
Anatoly Bugakov
  • 772
  • 1
  • 7
  • 18
0
votes
1 answer

Dealing with clone

I'm trying to iterate over a VCF file and create vectors with the data to build a DataFrame. However, the rust compilator is raising an error saying that the borrowed value does not live enough. I'm cloning the value because, in that case, I'm…
Evandro Lippert
  • 336
  • 2
  • 11
0
votes
1 answer

Polars meaning of parallelization?

I'd like to use this package as data backend to expose an api/website with data analysis How parallelization is done in this package ? is it possible to control the resources consumed ? Br
Devyl
  • 565
  • 3
  • 8
0
votes
1 answer

Control the compression level when writing Parquet files using Polars in Rust

I found that by default polars' output Parquet files are around 35% larger than Parquet files output by Spark (on the same data). Spark uses snappy for compression by default and it doesn't help if I switch ParquetCompression to snappy in polars. I…
Benjamin Du
  • 1,391
  • 1
  • 17
  • 25
0
votes
1 answer

expanding.apply in polars

In pandas I could call data.expanding(min_periods=1).apply(lambda_func) to call a func on expanding or a cumsum-like view. How to do the same thing with polars? I could only find rolling_apply or apply.
Hakase
  • 211
  • 1
  • 12
0
votes
0 answers

Inserting row data as column data using Hashmaps and structs

I'm reading data from a message queue, deserializing it with Serde and storing it into structs which I want to store that timeseries data into polars. Reading this my understanding is that polars is built arround storing the data in columns so as I…
Mass
  • 1
  • 2
1 2 3
…
18 19