I am trying to generate mini batch data using a sampler with built-in support in DataLoader to support random batch training on large images,
dgl.dataloading.DataLoader( graph, valid_nids, sampler, batch_size=1024, shuffle=False, drop_last=False, num_workers=0, device=device )
I found that this class takes the graph object as the first parameter, but due to the large amount of node and edge data, I encountered an OOM problem when using the following method to build the graph.
graph = dgl.graph((u, v))
My dataset has about 40 million nodes, plus about 200GB of feature information. I don't have a distributed environment and a single machine has 60GB of memory. How can I successfully generate mini batch data for training.
