-1

I'm new to deep learning so if the question doesn't make sense plz correct me.

In traditional machine learning I know how to compare models and choose one of the as the best with the metrics I choose.

However, in deep learning, each model is build with different layers, so how can I control variables to determine which model is the best fairly? Or usually people don't compare in this way?

For example I have a sequential data, I can use both CNN and LSTM model, so should I compare model with only one layer of CNN and one layer of LSTM? After that I can add more layers or tuning my model?

Or someone can just tell me the process of how to compare and choose the best deep learning model with chosen metrics?

aliaaaa
  • 21
  • 4
  • It sounds like you are wondering less about what performance metrics to run and more about how to know which parts of the model to modify to improve perfomance? – xdhmoore Apr 19 '21 at 17:56
  • So for example I choose accuracy as the comparison metrics, and I try to fairly compare the results using CNN and LSTM, so how can I achieve this? Since deep learning model can add more layers by myself? – aliaaaa Apr 19 '21 at 18:08
  • Perhaps looking at both accuracy and speed? I'm not sure what you mean by "fairly". I'm not sure that same "number of layers" equals a "fair" comparison if they are different kinds of models. But if you add more layers usually you slow down a little and that's often the important consideration. – xdhmoore Apr 19 '21 at 18:25
  • So for example I don't know a two layer CNN comparing to a one layer LSTM is reasonable or not? – aliaaaa Apr 19 '21 at 18:49

1 Answers1

0

For sequential data as you mention, Number of layers has nothing to do with the comparison of two models. At certain number of layers your accuracy will start to decrease because of overfitting. Comparing 1 layer of CNN with 1 layer of LSTM is not a correct approach.

You need to check the following factors for comparison like accuracy,precision,recall,f1-score depending on your application objective.

For example , if you are working on the language translation data LSTM would be better choice, because it over comes the problem of vanishing gradient.

Kiran G
  • 21
  • 3
  • But for example in the case I have model with 1 layer CNN with average accuracy higher than model with 1 layer LSTM, can I say I choose CNN because it perform better? – aliaaaa Apr 19 '21 at 20:57
  • No, you cant. There may be possibility that multiple layers of LSTM can out perform CNN with one or multiple layers. If you only want to use one layer, then , in that case you can choose CNN. But also check the test accuracy for both of the model. – Kiran G Apr 19 '21 at 21:13
  • So according to my understanding, we cannot say which deep learning model is better than the other? Like in traditional ml model comparison, we choose several model to run and compare the test results to select the best model. For deep learning model we can't make this comparison or chose? – aliaaaa Apr 19 '21 at 21:52