DataLoader is a generic utility to be used as part of your application's data fetching layer to provide a consistent API over various backends and reduce requests to those backends via batching and caching.
GitHub: dataloader
Questions tagged [dataloader]
430 questions
1
vote
0 answers
something wrong in custom pytorch dataset class for .h5 files
class HDF5Dataset(torch.utils.data.Dataset):
def __setup_files(self):
files = glob.glob(os.path.join(self.dir_path,'**/*.h5'))
return files
def __init__(self, dir_path, IMG_SIZE):
self.dir_path = dir_path
…
Jinhoon Jeong
- 53
- 5
1
vote
1 answer
Efficiently resolving belongs_to relationship in elixir dataloader?
Is it possible to use elixir dataloader to query a belongs_to relationship efficiently? It seems that the load is querying all of the items it needs, but the get is returning the first value of the loaded items regardless of which single item it…
Craig
- 139
- 1
- 5
1
vote
2 answers
Print random sample from dataloader in PyTorch
I have a certain dataset loaded into a dataloader. For example, if I wanted to save 100 images from this dataloader, how should I iterate over the dataloader to save them?
CaxiomX20
- 11
- 1
- 2
1
vote
0 answers
How to order slice of pointers using gqlgen and dataloaden (graphql dataloader) if some keys are NULL?
I have these entities:
Golang struct:
type Player struct {
ID int
CreatedAt time.Time
City City
CityID int
Team *Team
TeamID *int
Score int
}
GraphQL schema:
type Player {
id: ID!
…
Fred Hors
- 3,258
- 3
- 25
- 71
1
vote
1 answer
How to deal with readonly arguments from DataLoader - (GraphQL + Apollo in NextJS)
I have a loader function in order to fetch chefs from a database. So that loader receive an array of ids and what's is strange is that the ids have a readonly type.
When I try to pass that read-only type to the database query, it gives an…
Laura Beatris
- 1,782
- 7
- 29
- 49
1
vote
2 answers
next(iter(..)) function generate an error
I apply this code lines in my model
train_data_loader = create_data_loader(df_train, tokenizer, MAX_LEN, BATCH_SIZE)
data =next(iter(train_data_loader))
but I got this error
TypeError Traceback (most recent call…
AFB
- 83
- 1
- 3
- 10
1
vote
2 answers
GraphQL and Data Loader Using the graphql-java-kickstart library
I am attempting to use the DataLoader feature within the graphql-java-kickstart library:
https://github.com/graphql-java-kickstart
My application is a Spring Boot application using 2.3.0.RELEASE. And I using version 7.0.1 of the…
Dan DaLuga
- 85
- 2
- 10
1
vote
1 answer
Blazegraph Data Loader
I try to import some triple into the Blazegraph data base, whereas all triple w/o data type get loaded.
But triples having e.g. ^^xsd:date format get not loaded?!
Does anyone know how to deal with that issue?
Examples not being…
Spooky Fox
- 41
- 1
- 4
1
vote
2 answers
How to use dataloader?
Im trying to figure this out.
I want to get all my users from my database, cache them
and then when making a new request I want to get those that Ive cached + new ones that have been created.
So far:
const batchUsers = async ({ user }) => {
…
CodingLittle
- 1,761
- 2
- 17
- 44
1
vote
2 answers
Can I use data loader without batching
What I am really interested in with data-loader is the per request caching. For example say my graphql query needs to call getUser("id1") 3x. I would like something to dedupe that call.
However it seems like with data-loader I need to pass in an…
Robert Lemiesz
- 1,026
- 2
- 17
- 29
1
vote
2 answers
How to change DataLoader in PyTorch to read one image for prediction?
Currently, I have a pre-trained model that uses a DataLoader for reading a batch of images for training the model.
self.data_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=False,
num_workers=1,…
Hamid R. Darabi
- 109
- 2
- 10
1
vote
1 answer
RuntimeError in Pytorch when increasing batch size to more than 1
This code for my custom data loader runs smoothly with batch_size=1, but when I increase batch size I get the following Error:
RuntimeError: Expected object of scalar type Double but got scalar type Long for sequence element 1 in sequence argument…
Bilal
- 3,191
- 4
- 21
- 49
1
vote
1 answer
graphql sequlize dataloader issue, Cannot return null for non-nullable field
schema for ip address:
import { gql } from 'apollo-server-express';
module.exports = gql`
extend type Query {
# allTxn: [SubscrTxn]
cblockIp(ip_key: Int!): CblockIp
}
type CblockIp {
ip_key: Int!
…
Tornado
- 53
- 7
1
vote
1 answer
TypeError: Cannot handle this data type
Trying to put the saliency map to the image and make a new data set
trainloader = utilsxai.load_data_cifar10(batch_size=1,test=False)
testloader = utilsxai.load_data_cifar10(batch_size=128, test=True)
this load_cifar10 is torchvision
data =…
user11146622
1
vote
1 answer
Pytorch Dataloader for Image GT dataset
I am new to pytorch. I am trying to create a DataLoader for a dataset of images where each image got a corresponding ground truth (same…
yliats
- 93
- 2
- 10