I would like to train a simple classifier in C++, very much in the fashion of the C++ mnist example, where however my data is not stored on the HD but already loaded to the memory, say to an mxnet NDArray. In Python for this purpose one has the handy NDArrayIter, c.f. Module tutorial.
Is there such an NDArray iterator for C++?
Browsing through the code I found that all possible MXDataIter's can be read out from MXListDataIters and MXDataIterGetIterInfo:
#include "mxnet-cpp/io.h"
using namespace std;
using namespace mxnet::cpp;
int main(int argc, char** argv) {
Context ctx = Context::cpu(); // Use CPU
mx_uint num_data_iter_creators;
DataIterCreator *data_iter_creators = nullptr;
int r = MXListDataIters(&num_data_iter_creators, &data_iter_creators);
CHECK_EQ(r, 0);
cout << "num_data_iter_creators = " << num_data_iter_creators << endl;
//output: num_data_iter_creators = 8
const char *name;
const char *description;
mx_uint num_args;
const char **arg_names;
const char **arg_type_infos;
const char **arg_descriptions;
for (mx_uint i = 0; i < num_data_iter_creators; i++) {
r = MXDataIterGetIterInfo(data_iter_creators[i], &name, &description,
&num_args, &arg_names, &arg_type_infos,
&arg_descriptions);
CHECK_EQ(r, 0);
cout << " i: " << i << ", name: " << name << endl;
}
MXNotifyShutdown();
return 0;
}
which yields the eight MXDataIter()'s:
num_data_iter_creators = 8
i: 0, name: ImageDetRecordIter
i: 1, name: CSVIter
i: 2, name: ImageRecordIter_v1
i: 3, name: ImageRecordUInt8Iter_v1
i: 4, name: MNISTIter
i: 5, name: ImageRecordIter
i: 6, name: ImageRecordUInt8Iter
i: 7, name: LibSVMIter
So it appears to me that for C++ there is no NDArray iterator and that the easiest solution would be to write my data to csv-files only to then load it again into a MXDataIter(CSVIter). Another possibility would be to manually split the data into batch NDArray's and feed these into the training, but this also feels clumsy.
