0

I’m just getting started with dl4j. After going through some of the examples I thought I would have a go at implementing a simple convolutional network to classify a subset of images from the Google Quick, Draw! Dataset. The DataIterators I have been using from the examples are customised for specific datasets (MNIST, CIFAR etc.).

How would I create a custom DataIterator that converts the bunch of .npy files into a format that I can load into my network. More importantly, is this the correct approach?

J Adamson
  • 417
  • 1
  • 5
  • 13

1 Answers1

-1

Deeplearning4j can parse numpy output. You can use the various numpy methods here: https://github.com/eclipse/deeplearning4j/blob/master/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/factory/Nd4j.java#L5654

Example: INDArray arr = Nd4j.createFromNpyFile(yourFile);

A dataset is just 2 ndarrays so you can create a dataset from: DataSet d = new DataSet(features,labels)

A datasetiterator just returns datasets. What you would have to do is implement the dataset iterator interface reading in datasets using the create from npy file above.

Adam Gibson
  • 3,055
  • 1
  • 10
  • 12