suggest deep learning model for text topic classification

Question

I have a dataset consisting of two columns [Text, topic_labels]. Topic_labels are of 6 categories for ex: [plants,animals,birds,insects etc]

I want to build deep learning-based models in order to be able to classify topic_labels. so far I have implemented both supervised[SVM, Logistic] & unsupervised [topic-LDA, Guided-LDA] approaches in a traditional way by applying both Word2Vec & TF-IDF but I wanted to implement state-of-the-art deep learning classification techniques for the text data?

Suggest me the best deep learning model for text topic classification

Answer 1

Here are some of my suggestions. Since you have a dataset consisting of two columns [Text, topic_labels], and Topic_labels are of 6 categories for ex: [plants,animals,birds,insects etc] only. This is a relatively small task. I recommend you choose the model that focuses on accuracy, rather than speed and memory. Accuracy is defined as follows.

TP, FP, TN, FN denote true positive, false positive, true negative, and false negative.

I recommend the models stated in this paper. In general there are two categories:

1. Rule-based methods. Rule-based methods classify text into different categories using a set of pre-defined rules, and require a deep domain knowledge like linguistic. Rule-based approaches classify text into organized groups by using a set of linguistic rules. One of the most successful rule-based algorithms in Topic classification is transformation-based learning (TBL).
2. Machine learning (data-driven) based methods

Since you mentioned deep learning, you want the second category. In the second category, an accurate method is the Feed-forward networks. Even though they are quite simple, they have achieved high accuracy on many Text-classication (or Topic-classication if you will) benchmarks.

Feed-forward networks view text as a bag of words. For each word, they learn a vector representation using an embedding model such as word2vec or Glove, take the vector sum or average of the embeddings as the representation of the text, pass it through one or more feed-forward layers, known as Multi-Layer Perceptrons (MLPs), and then perform classification on the final layer’s representation using a classifier such as logistic regression, Naïve Bayes, or SVM.

However, if you want more fancy ones, and the latest state-of-the-art one, you can read the following table.

The most accurate one is XLNet-Large (ensemble) among all the instances in the benchmarking.

Answer 2

For the state of art model in Deep learning you can go for BERT models from hugginface.co. It is easier to implement as well as will give you good accuracy but they are memory inefficient models.