Why different deep learning network predict at almost the same speed?

Question

I tried different deep learning network structures. Some of them are very deep(more than 40 layers), others are less than 20 layers. Apparently the costs of training are quite different. However, these networks classify images at almost the same speed. Is it the case or I missed something?

Answer 1

The fact that they seem to have the same speed at processing images is due to the fact that you are just processing one image, and therefore both times will be small. However, if you had to process a large number of images, then the small difference of time in processing each image would cause a large time difference at the end.

Also keep in mind that training is much more time consuming than just processing an image, because in the training phase the backpropagation algorithm has to be computed to update the weights, so the difference in speed is more evident in the training phase.