Stack Overflow
Stack Overflow

Questions tagged [fsspec]

Filesystem interfaces for python

A low level IO library used by many pydata packages. See the docs https://filesystem-spec.readthedocs.io/en/latest/

39 questions
0
votes
1 answer

Does fsspec support virtual filesystems such as pyfileysystem

One of pyfilesystem's main feature is virtual filesystems. E.g. home_fs = open_fs('~/') projects_fs = home_fs.opendir('/projects') I think that is a great feature and was hoping that fsspec has something similar. But I couldn't find an example and…
Juergen
  • 699
  • 7
  • 20
0
votes
1 answer

import fsspec throws error (AttributeError: 'EntryPoints' object has no attribute 'get')

import fsspec throws error (AttributeError: 'EntryPoints' object has no attribute 'get') []
Abhijit Das
  • 111
  • 2
  • 8
0
votes
1 answer

FSSpec Error Handling in Python - Timeout Error

I am trying to get Terraclimate Data from Microsoft Planetary and facing time out error. Is there a possiblity of increasing the timeout time ? Please find the code below and the error I am facing. I am using fsspec and xarray for downloading…
m3tal_slayer
  • 21
  • 6
0
votes
1 answer

TypeError: 'LocalFileOpener' object is not iterable

I have a huge dataset with millions of entries (It is a normal .csv file and I get no errors when I load it with pandas). Pandas struggles when trying to load the dataset (.csv), so I decided to use modin, which apparently allows you to use multiple…
CozyCode
  • 484
  • 4
  • 13
0
votes
1 answer

Deleting a file from Google Cloud using fsspec

I am currently using fsspec in order to read and write files to my google cloud buckets with the code as shown below: with fs.open(gcs_file_tmp, "rb") as fp: gcs_file_content = fp.read() Now I want to remove a file from a GCS bucket, but I…
SDG
  • 2,260
  • 8
  • 35
  • 77
0
votes
1 answer

No Such File Error When Trying to Create Local Cache of s3 Object

I'm trying to use Fsspec to create a local cache of a data file store in a public access bucket on AWS s3. The public access bucket is located here. It is 100% necessary for me to do this in local file cache because this intended to scale and I…
Zach Rieck
  • 419
  • 1
  • 4
  • 23
0
votes
3 answers

Using fsspec at pandas.DataFrame.to_csv command

I want to write the csv-file from pandas dataframe on remote machine connecting via smtp-ssh. Does anybody know how add "storage_options" parameter correctly? Pandas documentation says that I have to use some dict as parameter's value. But I don't…
Юрий Шахов
  • 1
  • 1
  • 2
  • 5
0
votes
1 answer

Add entry_points to setuptools in package being install

I have a python package fsspec. This library provides a way to register external filesystem (backend integration). One way to register the required filesystem is to add it to the entry_points in setuptools. To accomplish the task, we can manually…
Manjur Ansari
  • 59
  • 8
0
votes
1 answer

how to add fsspec.open_local in backend of xarray

Following the Q here xarray read remote grib file on s3 using cfgrib How would I convert the following code to use in the backend_kwargs of xarray's open_dataset. import fsspec import xarray as xr uri =…
Ray Bell
  • 1,508
  • 4
  • 18
  • 45
1 2
3