Stack Overflow
Stack Overflow

Questions tagged [feather]

Feather is a file format for storing data frames. It allows fast data exchange between Python and R.

147 questions
3
votes
2 answers

Creating a python desktop app to navigate and view S3 files without using aws credentials

I have a use case where I need to make a S3 navigator which should allow users to navigate s3 files and view them without giving any sort of aws access. So users need not have aws credentials configured on their systems. The approach I tried is to…
Naxi
  • 1,504
  • 5
  • 33
  • 72
3
votes
1 answer

Faster serializations (pickle, parquet, feather, ...) than json in plotly dash Store?

Context In a dashboard using plotly Dash I need to perform an expensive download from DB only when a component (DataPicker with the period to consider and so to be downloaded from DB) is updated and then use the resulting DataFrame with other…
user11696358
  • 356
  • 1
  • 15
3
votes
1 answer

How to correctly read an Apache Arrow Feather file produced by pyarrow?

I have been unsuccessful to read an Apache Arrow Feather with javascript produced by a python script javascript library of Arrow.. I am using pyarrow and arrow/js from the Apache Arrow project. I created a simple python script to create the Feather…
ToniR
  • 33
  • 7
3
votes
1 answer

How to read column names and metadata from feather files in R arrow?

The (now-superseded) stand-alone feather library for R had a function called feather_metadata() that allowed to read column names and types from feather files on disk, without opening them. This was useful to select only specific columns when…
MatteoS
  • 745
  • 2
  • 6
  • 17
3
votes
0 answers

Loading feather files from s3 with dask delayed

I have an s3 folder with multiple .feather files, I would like to load these into dask using python as described here: Load many feather files in a folder into dask. I have tried two ways both give me different errors: import pandas as pd import…
Dean
  • 105
  • 1
  • 6
3
votes
1 answer

Can I convert a .rda file to a pandas dataframe in python without using R?

I'm practicing my Python, specifically my numpy and pandas. I have some data (not mine) in .rda format that I want to import into python as a dataframe. However I don't use R, so I'm wondering if I can do this without fiddling around with the base…
artnim
  • 33
  • 3
3
votes
0 answers

Is it possible to save and read feather file in compressed zip format in R?

I need to save feather data in a more compact format. After making several tests I found that after using the "zip" function the size of the "feather" data file reduce up to 90%. library(feather) library(zip) # Write data in feather…
Andrii
  • 2,843
  • 27
  • 33
3
votes
1 answer

How to compress and decompress Arrow or Feather file?

I plan to change data file format from parquet to feather. Parquet has compression options(lz4, etc) and I have used them. But I can not find them in feather or Arrow file. Is compression not supported?
makepossible99
  • 47
  • 5
3
votes
1 answer

Error using the feather format package

I am having trouble using the feather format to export data. I have installed the package using: conda install feather-format -c conda-forge however, when I use pd.to_feather() I get an error telling me to install the package (which is already…
Ame.Lia
  • 43
  • 3
3
votes
1 answer

How to read binary compressed SAS files in chunks using panda.read_sas and save as feather

I am trying to use pandas.read_sas() to read binary compressed SAS files in chunks and save each chunk as a separate feather file. This is my code import feather as fr import pandas as pd pdi = pd.read_sas("C:/data/test.sas7bdat", chunksize =…
xiaodai
  • 14,889
  • 18
  • 76
  • 140
3
votes
0 answers

Tibble with Date column is not identical to itself after writing and reading with feather

Let's consider the tibble x <- tibble::tibble(start_date = lubridate::ymd(NA, NA, "2016-01-01", "2018-04-01", NA)) Obviously it is identical to itself testthat::expect_identical(x, x) But if i write the file to disk and then I read it then I get…
amarchin
  • 2,044
  • 1
  • 16
  • 32
3
votes
1 answer

iOS 9.3 UIGraphicsImageRenderer showing nil any other option for this?

I'm want apply feather effect to the image when cropped but want i render the image using UIGraphicsImageRenderer the render is nil(iOS 9.3) is there any other option for rendering the image in iOS 9.3 This is my code: UIImage* Img =…
Mitesh Varu
  • 123
  • 1
  • 10
2
votes
2 answers

Best way to store many pandas dataframes in a single file

I have 20,000 ~1000-row dataframes, each of which has a name, in a 170GB pickle file at the moment. I'd like to write these to a file these so I can load them individually, by name. I won't need to query for subsets of these dataframes, only read…
Alex Lenail
  • 12,992
  • 10
  • 47
  • 79
2
votes
1 answer

How to convert data in Polars?

I used .write_ipc from Polars to store as a feather file. It turns out that the numerical strings have been saved as integers. So I need to convert the columns with integers to strings either before saving as feather or after reading from feather.…
fvg
  • 153
  • 3
  • 9
2
votes
0 answers

Saving multiple pandas dataframes to one file and then reading them back

I am working on a python app and I am trying to logistically plan how saving/loading files will work In this app multiple data sheets will be loaded and they will all be used in some capacity, the problem I am having is that I want users to be able…
Coolroo
  • 57
  • 6
1 2
3
…
9 10