1

Currently I have pytorch tensors with shape (batch_size, height, width, channel_size) and I want to convert it to a mini-batch described here. My current idea is to convert each example from tensor representation to graph representation separately and group them together. I want to do all these without involving file save/load as it will surely hinder the speed (I notice that Creating “In Memory Datasets” does it this way).

Yet I didn't find any function for the example grouping part. Could anyone please help give a plausible workflow for it, and is there any smarter way for this convertion, from tensor to mini-batch for pytorch-geometric?

Sparky05
  • 4,692
  • 1
  • 10
  • 27
Original-Thunderbird
  • 11
  • 1
  • 2
    So you would like to convert `(batch_size, height, width, channel_size)` to something like `(nb_mini_batches, mini_batch_size, height, width, channel_size)`? – Ivan Jan 09 '21 at 10:59
  • @Ivan Not really. The definition of torch_geometric.data.Data (for graph representation of one example) can be seen [here](https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html#mini-batches). I currently plan to first convert each example with shape [height, width, channel_size] to such a representation, but any smarter way is appreciated. – Original-Thunderbird Jan 09 '21 at 13:08
  • Could you provide a minimal example with the desired result? – Ivan Jan 09 '21 at 13:12

1 Answers1

1

I think I'm experiencing a similar question with you. If I understand your question correctly, your want to commit following transformation

Input: Tensor = [#batch,#vertex,#feature]
Output: torch_geometric.data.BatchData = Large tensor

My implementation is:

x = DataLoader([Data(x,edge_index=edges,num_node=#node) for x in x],batch_size=#batch)
data = next(iter(x))
Haowei Lou
  • 11
  • 1