DataLoader is a generic utility to be used as part of your application's data fetching layer to provide a consistent API over various backends and reduce requests to those backends via batching and caching.
GitHub: dataloader
Questions tagged [dataloader]
430 questions
0
votes
1 answer
Dataloader caches across context creations (graphQl ApolloServer)
I have integrated DataLoaders into my graphQL requests, but for some reason the cache is memoized across requests. I can read that passing a function to context in the ApolloServer config should create a new context on each request, but for some…
Thjen
- 527
- 1
- 9
- 18
0
votes
1 answer
Show specific columns with dataloader-sequelize in nested tables
Currently I have some models. I'm using graphql with dataloader-sequelize and it works fine as long as I show associated tables without third level.
My models:
"articulo.js"
'use strict';
module.exports = (sequelize, DataTypes) => {
const Articulo…
Andynedine
- 441
- 5
- 20
0
votes
1 answer
MongoDB batch get for DataLoader
I actually resolved this issue, however, posting here as a question in hope someone will find the solution useful. I was looking for a purely MongoDB query/aggregation that would serve as a batch getter for FB's DataLoader. I didn't want to .map()…
gazorpazorp
- 468
- 3
- 13
0
votes
1 answer
How can I get the IDs of specific items in a Pytorch dataloader-based dataset with a query?
I have a large dataset (approx. 500GB and 180k data points plus labels) in a Pytorch dataloader. Until now, I used torch.utils.data.random_split to split the dataset randomly into training and validation. However, this lead to serious overfitting.…
Michael
- 280
- 2
- 13
0
votes
0 answers
How to process dataset from loader in the same order at every training
I would like to know how to process the indices that I have in my data loader always at the same order for every training. I want to do this because it takes a lot of time for my code to save visual outputs of the results on validation set, so I…
luigui2906
- 57
- 6
0
votes
1 answer
GraphQL - does context propagate to downstream resolvers?
If you pass a modified context to a GraphQL resolver does this propagate to all downstream resolvers? Is this specified in the GraphQL specification or implementation specific?
To clarify with an example say I have a query like the following
{
…
Jordan
- 5,085
- 7
- 34
- 50
0
votes
0 answers
subrandomsapler dataloader pytorch lengh does not seems right
split = int(len(train_dataset) * 0.8)
print(split)
index_list = list(range(len(train_dataset)))
train_idx, valid_idx = index_list[:split], index_list[split:]
print(len(train_idx),len(valid_idx))
48000 12000
I got this 48000 and…
user11173832
0
votes
1 answer
pytorch ImageFolder datasize does not match with the original imgs number?
I have 21,522 items, totalling 609.7 MB in train folder
train
-1(3600 png)
-2(3600 png)
.
.
-6
train_trans = transforms.Compose([
vision.transforms.Resize(target_size),
vision.transforms.RandomHorizontalFlip(),
…
user11173832
0
votes
1 answer
Elixir + Absinthe + Ecto + Dataloader – filter by multiple fields
Using Elixir/Absinthe/Ecto/Dataloader, how do you query/filter a source by multiple fields?
Example:
Let's say you would want to filter a schema (and dataloader source) named User by two fields, one named is_admin (only true values) and one group_id…
gunnar2k
- 185
- 1
- 13
0
votes
1 answer
Is it possible to drop_last in reverse with PyTorch's DataLoader?
I'm currently trying to make a custom PyTorch DataLoader.
I'm aware that setting drop_last=True when first declaring the DataLoader object tells the object to drop the last incomplete batch if the size is not appropriate. However, I was wondering if…
Sean
- 2,890
- 8
- 36
- 78
0
votes
0 answers
Convert Dataloader Dictionary to Pytorch Tensor
I am trying to apply an autoencoder to a custom dataset in Pytorch using the following tutorial as a guide.
https://pytorch.org/tutorials/beginner/data_loading_tutorial.html
My dataset consists of images and values associated with those images.
I…
Random Engineer
- 1
- 1
- 2
0
votes
0 answers
Building a dataset with dataloader pytorch getting error cannot import name 'read_data_sets'
Loading data into dataset using pytorch dataloader.
Getting error cannot import name 'read_data_sets'
Tried searaching for results from similar issues.
If there is confusion about file instead of module and it can't find read_data_sets in your file …
mainautumn
- 1
- 2
0
votes
1 answer
"cannot unpack non-iterable function object" : trying to run a code through tensorflow
while running this code I am getting error
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data
this should run fine .
Anybody know how to resolve this?
Please help me
BraveStone9
- 53
- 10
0
votes
1 answer
How to use DataLoader with Mongoose
I'm trying to build the following use case of DataLoader together with Mongoose:
export const PurchaseOrderType = new GraphQLObjectType({
name: "PurchaseOrder",
description: "PurchaseOrder",
interfaces: () => [NodeInterface],
…
Mendes
- 17,489
- 35
- 150
- 263
0
votes
1 answer
Memory error when iterate over two dataloaders simultaneously in pytorch
I am trying to train my model using 2 dataloaders from 2 different datasets.
I found how to set up this by using cycle() and zip() because my datasets are not the same length from here: How to iterate over two dataloaders simultaneously using…
afroditi
- 307
- 1
- 3
- 13