My Google colab crashes immediately as soon as it starts training on tiny-imagenet
with 0.1 million images and 200 classes of size 64*64
Colab log shows
WARNING:root:kernel 1fe0be22-c98a-4519-a16a-69c9fb4be1da restarted
KernelRestarter: restarting kernel (1/5), keep random ports
tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10754 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
I am using model.fit_generator
with batch size (tried from 32 till 1024) and size of image ( tried from 16 till 64) but nothing works.
I tried resnet-18
architecture with (1.8*10^9 params) as well as custom model too with 0.8 million params but in vain.
I am pasting the link to my colab in case anybody needs some other info https://colab.research.google.com/drive/1QG1mg1zOn6gZaaSv4rrI4F6erdxsxQ8V#scrollTo=Uy0M-VDHivOX