Stack Overflow
Stack Overflow

Questions tagged [feather]

Feather is a file format for storing data frames. It allows fast data exchange between Python and R.

147 questions
1
vote
1 answer

Dataframe with Array Column to New Dataframe

Actually i have an feather file that im loading to an Dataframe. And this Dataframe have a column with languages in each row. Like the abone one: student_id name created_at languages 1 Foo 2019-01-03 14:30:32.146000+00:00 [{'language_id':…
Hudson Medeiros
  • 301
  • 2
  • 11
1
vote
0 answers

populating elements of dict using pandas.read_pickle() results in killed python process

On an Ubuntu 18.04.5 image running on AWS, I've noticed that attempting to populate a dict with multiple (7, in my case) dataframes loaded via pandas.read_pickle(), e.g., using something like import pathlib import pandas as pd df_dict = {} base_dir…
lebedov
  • 1,371
  • 2
  • 12
  • 27
1
vote
1 answer

Is there an efficient way of changing a feather file to a parquet file?

I have a big feather file, which I want to change to parquet, so that I can work with Pyspark. Is there a more efficient way of change the file type than doing the following: df = pd.read_feather('file.feather').set_index('date') df_parquet =…
TiTo
  • 833
  • 2
  • 7
  • 28
1
vote
0 answers

Panda's chunksize equivalent in feather file?

I am trying to convert my code which used csv/txt file to feather file code. I read the file like df = pd.read_csv(r"C:\Users\Saad.Saleem\Downloads\message.txt", engine='python', sep=';', encoding='utf-8', chunksize=10) but I can't find chunksize…
Saad Saleem
  • 75
  • 8
1
vote
0 answers

"Process finished with exit code 135 (interrupted by signal 7: SIGEMT)" error while reading feather file

I have some Python code that reads in a number of data files and performs some cleaning steps on them. I've been getting a very strange error message during the read_feather function call: Process finished with exit code 135 (interrupted by signal…
Adrian Keister
  • 842
  • 3
  • 15
  • 33
1
vote
0 answers

Read in feather file directly from GitHub in R

How can I read in a .feather file from the web (e.g. GitHub) in R? I can read formats as .csv or .dta from GitHub directly as raw # CSV coursedata <- read.csv(file =…
Marco
  • 2,368
  • 6
  • 22
  • 48
1
vote
0 answers

Does the R arrow package have anything like the random access capability of the fst package?

Our team is looking to integrate more of our R and python work. One part of this effort has been trying to move from fst files (using the package fst), which as far as I know cannot be read in python without interfacing with R (Is it possible to…
Andreas
  • 210
  • 2
  • 9
1
vote
1 answer

filter date from a string JULIA

I have the next format for my date in my dataframe: Typeof(DateTime("2021-12-17T06:00:00")) Feather.Arrow.Timestamp{Microsecond} I want to filter the data by date, but I can't because of the type. I tried to chop it, but again because of the type I…
Samantha Escarlet Cassab Balbu
  • 11
  • 1
1
vote
1 answer

How do you write a .feather file into GCS?

Previously worked on .csv files which was straightforward to upload to GCS For csv I would do the following, which works: blob = bucket.blob(path) blob.upload_from_string(dataframe.to_csv(), 'text/csv') I am trying to do the same i.e. write the…
Kyo
  • 27
  • 4
1
vote
1 answer

2 .feather files with same data, completely different sizes?

I have 2 feather files based on the same data. The only difference is the way the data is obtained. File 1 has a list of queries, broken out by month, that are each saved as individual files. Then each file is read into a dictionary and concatenated…
Mofongo
  • 131
  • 8
1
vote
0 answers

nRF52840 is ignoring certain setWriteCallback() calls

so I just started experimenting with the Adafruit Feather nRF52840 board and have set up a working BLE connection between the board and another device. Usually, everything works fine by sending data from the other device to the nRF52840, but…
Acimaz
  • 171
  • 1
  • 14
1
vote
0 answers

R session aborted - Windows - Large datasets

I am trying to merge several databases from various years (2009 to 2018) with about 800,000 observations each into a single database in feather format but every time I try to do this I get an "R session aborted" message I have already tried to clean…
user16019366
1
vote
0 answers

Reading a feather file from S3 using api gateway proxy for S3

I am trying to read a feather file from an S3 bucket using api gateway proxy for s3 with no luck. Tried everything but every time I get below error. ine 239, in read_table reader = _feather.FeatherReader(source, use_memory_map=memory_map) File…
Naxi
  • 1,504
  • 5
  • 33
  • 72
1
vote
1 answer

how to enable memory mapping while reading feather file in c++

While reading the same feather file in python and in c++,in python, the function pyarrow.feather.read_table() performs expectionally well then the API i am using for c++. When i investigated further, i found out that the main difference is because…
Sourabh Kulhari
  • 33
  • 1
  • 6
1
vote
0 answers

Segmentation fault while reading feather file containing columns with datatype arrow::large_utf8()

I had written c++ code to read feather file and insert the data into a arrow::Table, but it gives segmentation fault if the file contains any column with datatype arrow::large_utf8. It gives segfault for this datatype only, there are no errors for…
Sourabh Kulhari
  • 33
  • 1
  • 6
1 2 3
…
9 10