Learning rate ,Loss and Batch size

Question

Is Loss dependent upon learning rate and batch size. For .e.g if i keep batch size 4 and a learning rate lets say .002 then loss does not converge but if change the batch size to 32 keeping the learning rate same , i get a converging loss curve. Is this okk?

Answer 1

I would say that the loss is highly dependent on what parameters you use for your training. On the other hand, I would not call it a dependency in terms of a mathematical function but rather a relation.

If your network does not learn you need to tweak the parameters (architecture, learning rate, batch size, etc.).

It is hard to give a more specific answer to your question. What parameters that are ok are depending on the problem. However, if it converges and you can validate your solution I would say that you are fine.