Stack Overflow
Stack Overflow

Questions tagged [dataloader]

DataLoader is a generic utility to be used as part of your application's data fetching layer to provide a consistent API over various backends and reduce requests to those backends via batching and caching.

GitHub: dataloader

430 questions
1
vote
1 answer

how to adjust dataloader and make a new dataloader?

let say I have a data loader of cifar10 if I want to remove some value from the dataloader and make a new dataloader how should I do it? def load_data_cifar10(batch_size=128,test=False): if not test: train_dset =…
user11173832
1
vote
1 answer

Trouble crating Dataset and Dataloader for HDF5 file in Pytorch: not enough values to unpack (expected 2, got 1)

After using datasets from Torchvision, I am trying to load in Pytorch a HDF5 file instead, with no success. I read I should define my own Dataset and Dataloader classes containing getitem to enable indexing and len to return the length of the…
teff ros
  • 29
  • 1
  • 6
1
vote
1 answer

Run-time error for (allegedly) empty batches in Pytorch

(Updated with further insights on the problem) I have a dataset of 3000 images which gets into a DataLoader with the following lines: training_left_eyes = torch.utils.data.DataLoader(train_dataset, batch_size=2,shuffle=True,…
Carlos Hernandez Perez
  • 331
  • 4
  • 16
1
vote
0 answers

Caching mechanism of DataLoader in graphql for DotNetCore

//Sample Query Field() .Name("xyz") .Argument>("param1", "desc")//Query parameter .Argument>("param2", "desc")//Query parameter …
Niraj Bhattad
  • 121
  • 1
  • 1
  • 6
1
vote
1 answer

How to use DataLoader for PyTorch on iPython Console of Spyder

I check this Tutorial and can't figure out a way to actually use my DataLoader to train a ANN. When iterating over my DataLoader a cmd prompt pops up and immediately closes itself, afterwards nothing happens. My original data are both…
Quastiat
  • 1,164
  • 1
  • 18
  • 37
1
vote
2 answers

DataLoader messing up transformed data

I am testing the MNIST dataset in Pytorch, and after I apply a transformation to the X data, it seems the DataLoader puts all values out of the original order, potentially messing up the training step. My transformation is to divide all values by…
Denny Ceccon
  • 145
  • 9
1
vote
1 answer

Matplotlib Pylot - Images are being displayed in low resolution (pixel to pixel)

When I display some samples photos from the dataset I use, the previews of images are displayed in low resolution (they look like very low-resolution photos). How I can I display the images without losing their resolutions? Here are my…
talha06
  • 6,206
  • 21
  • 92
  • 147
1
vote
0 answers

Using DataLoader together with Relay cache

I've developed a graphql server (using graphql-yoga), my GraphQL client is Relay and I've set up a cache in Relay Network layer as well (see the doc). My GraphQL server is just a layer on top of REST API. Does it make sense for me try to add…
A. Karnaval
  • 727
  • 2
  • 8
  • 12
1
vote
1 answer

Matching the DataLoader API using SQL only

One of the examples for using DataLoader with Knex shows something this: user: new DataLoader(ids => db.table('users') .whereIn('id', ids).select() .then(rows => ids.map(id => rows.find(x => x.id === id)))), The map there is so that the…
Jure Triglav
  • 1,862
  • 18
  • 22
0
votes
0 answers

What could be the reason of bulk load running from Dbeaver during salesforce refresh without fail?

Can I get some help on the below points please I am able to run bulk load using Dbeaver during instance refresh not able to understand what could be the reason if the salesforce service brought down then how this is being perform doesn't mean the…
khushi
  • 1
  • 1
0
votes
1 answer

I'm getting KeyError: 'bboxes' while using DataLoader, why that is happening?

I am working on image classification by using the PyTorch framework and struggling with the following error: KeyError: Caught KeyError in DataLoader worker process 0. Original Traceback (most recent call last): File…
ibrahim ahmethan
  • 17
  • 1
  • 6
0
votes
0 answers

Confusion in initialising GraphQL Dataloader in context

context: ({ req }) => { if (req) { return { ip: headers.userip, headers, userLanguage, decodedToken, dataLoaders: { seoDataLoader: createSeoDataLoader() } } } } Here I create a createSeoDataLoader instace for every request . now suppose if i have…
Jay Karavadra
  • 1
0
votes
1 answer

salesforce dataloader installation problem

I am learning salesforce. I am trying to install dataloader. I have downloaded and extracted the files in the zip file, into a local folder. But when I am trying to run the install.bat, I get this error. 2023-08-14 19:08:29,778 FATAL [main]…
santubangalore
  • 796
  • 1
  • 12
  • 25
0
votes
0 answers

ValueError: Expected input batch_size (1052) to match target batch_size (508) when fine tuning GPT 2 model

Hello there I'm attempting to train a GPT 2 model how to summarize passages without compromising their emotional impact. Consider summarizing a chapter from a book, but we want the reader to experience the same emotions as the chapter itself. I…
Damika
  • 622
  • 2
  • 8
  • 17
0
votes
0 answers

dataloader Pytorch - View Loaded Data

I am trying to run the code of this a Transformer paper. I am loading the data into the data loader, but I want to pass loop across the batches and view the data. I am new to dataloader and I know that it is a simple process, but I cannot seem to…
Sar99
  • 43
  • 5
1 2 3
…
28 29