DataLoader is a generic utility to be used as part of your application's data fetching layer to provide a consistent API over various backends and reduce requests to those backends via batching and caching.
GitHub: dataloader
Questions tagged [dataloader]
430 questions
0
votes
1 answer
Apollo graphQL : call dispatch from dataloader at the end of a query depth level
I recently implemented in nodeJS a dataloader for my graphQL API.
It works fine when I set the batchScheduleFn to a standard setTimeout for 100 ms for example, so I know everything works and fetch the data correctly.
What I want to do is to make the…
Bjornir
- 43
- 1
- 5
0
votes
1 answer
How tu upload JSON Array Object data via Bulk API
I have gone through salesforce documentation for bulk API 2.0 and it didn't support the Content-Type to be JSON
Now I have seen content type to be JSON in the bulk API 1.0 or simply bulk API.
but I haven't seen any documentation on how to achieve…
Malik
- 121
- 1
- 1
- 17
0
votes
1 answer
using dataloader to interface kafka data
i use dataloader to inferface the data in kafka and it doesnt work
here is my code
class kfkdataset(Dataset):
def __init__(self,consumer,image_size):
super(kfkdataset).__init__()
self.image_size=image_size
self.consumer = consumer
def…
yenan wu
- 1
0
votes
1 answer
Constant "potential loss of data" errors when importing Salesforce Dataloader CSV using SQL Import Tool
I have exported a list of accounts from Salesforce using their Dataloader tool. The output file is a CSV file. I have the table I want it imported into already created. I was using nvarchar(255) for all fields, but after I kept getting truncation…
Dizzy49
- 1,360
- 24
- 35
0
votes
2 answers
Custom Dataset, Dataloader, Sampler, or something else?
I'm working on a project that requires training a PyTorch framework NN on a very large dataset of images. Some of these images are completely irrelevant to the problem, and but these irrelevant images are not labelled as such. However, there are…
0
votes
1 answer
Pytorch transformation on MNIST dataset
I currently have a project with Weak Supervision where I need to put a "masking" in front of a dataset. My issue right now is that I don't exactly know how to do it. Let me explain further with some code and images.
I am using the MNIST dataset…
Kevin Laemers
- 5
- 1
- 4
0
votes
0 answers
Having problem with creating a dataloader class in PyTorch
I am building a neural network for Bengali numerical digit classification using PyTorch. I am facing difficulties building the dataset class to load my dataset using a data loader. I have a folder with all the images (numerical digits from 0-9) and…
aman.khashbar
- 1
- 2
0
votes
1 answer
Pytorch: Loading sample of images using DataLoader
I use standard DataLoader from torch.utils.data. I create dataset class and then build DataLoader this way:
train_dataset = LandmarksDataset(os.path.join(args.data, 'train'), train_transforms, split="train")
train_dataloader =…
Mikhail_Sam
- 10,602
- 11
- 66
- 102
0
votes
1 answer
mongoose findMany query with refs
I am new to mongoose I don't really find any reference how mongoose populate deals with findMany query having ref
for example I have a categories collection and a products collection as on product can be assigned to many categories that's why they…
user1592129
- 461
- 1
- 5
- 16
0
votes
1 answer
Error in Transforming custom Dataset in Pytorch
I was following this tutorial: https://pytorch.org/tutorials/beginner/data_loading_tutorial.html for making my own custom data loader for MuNuSeg Dataset but I am stuck a point. The dataloader is working fine but when I add transforms to it, I get…
Abhiraj Kanse
- 3
- 1
0
votes
1 answer
Pytorch Dataloader shuffle with multiple dataset
I'm trying to make custom Dataloader with multiple datasets.
My question is that if I use (shuffle = True) in the Dataloader option, is it possible to shuffle the same order in multiple Dataloader?
For example:
dataloader1: label = [5 , 4, 15,…
dlwowlb
- 3
- 4
0
votes
1 answer
Custom data loader for images and labels in pytorch. Difficulty with inputs in loss function
I was working on this https://www.kaggle.com/gti-upm/leapgestrecog data set lately. Its a hand gesture dataset and I was trying to make a classifier. Due to images available in different types of folder I made my on data loader. Here it is
class…
Ambesh Shekhar
- 3
- 2
0
votes
0 answers
SASxport to R: Errors while reading XPT SAS file
Anyone knows how to ignore/skip error while getting SAS export format file into R?
require(SASxport)
asc = SASxport::read.xport("..\\LLCP2018.XPT_", keep = cols)
Checking if the specified file has the appropriate header
Extracting data file…
Peter.k
- 1,475
- 23
- 40
0
votes
1 answer
Pytorch tensor data is disturbed when I convert data loader tensor to numpy array
I am using a simple train loop for a regression task. To make sure that regression ground-truth values are the same as what I expect in the training loop, I decided to plot each batch of data.
However, I see that when I convert the data loader’s…
BenyaminH
- 23
- 5
0
votes
0 answers
TypeError: object of type has no len() when using Pandas Dataframe
I tried the following code
import pandas as pd
import os
from torch.utils.data import Dataset
from torch.utils.data import Dataset, DataLoader
class ArticleDataset(Dataset):
def __init__(self):
# super().__init__()
df=…
Dilrukshi Perera
- 917
- 3
- 17
- 31