Matmul error when trying to predict a new text using skmultilearn.BinaryRelevance

Question

I am trying to create a small example of multi label text classification:

import skmultilearn
from sklearn.feature_extraction.text import TfidfVectorizer
import pandas as pd
from scipy.sparse import csr_matrix
from pandas.core.common import flatten
from sklearn.naive_bayes import MultinomialNB
from skmultilearn.problem_transform import BinaryRelevance

TRAIN_DATA = [

    ['Como efetuar uma conexão com MySQL usando PHP ?', ['desenvolvimento','banco']],
    ['Quais são os melhores clientes de VPN hoje em dia?', ['redes']],
    ['Qual é o equivalente ao tipo booleano no Oracle?', ['banco']],
    ['Como remover entidade indesejada da sessão do Hibernate?', ['desenvolvimento']],
    ['Como implementar o pool de conexão TCP em java?', ['desenvolvimento','redes']],
    ['Como posso me conectar ao banco de dados PostgreSQL remotamente de outra rede?', ['banco','redes']],
    ['Qual a função python para remover acentos em uma string?', ['desenvolvimento']],
    ['Como remover índices no SQL Server?', ['banco']],
    ['Como configurar o firewall com DMZ?', ['redes']]
]

data_frame = pd.DataFrame(TRAIN_DATA, columns=['text','labels'])
corpus = data_frame['text']
unique_labels = set(flatten(data_frame['labels']))
for u in unique_labels:
    data_frame[u] = 0
    data_frame[u] = pd.to_numeric(data_frame[u])
for i, row in data_frame.iterrows():
    for u in unique_labels:
        if u in row.labels:
            data_frame.at[i,u] = 1
tfidf = TfidfVectorizer()
Xfeatures = tfidf.fit_transform(corpus).toarray()
y = data_frame[unique_labels]
binary_rel_clf = BinaryRelevance(MultinomialNB())
binary_rel_clf.fit(Xfeatures,y)
predict_text = ['SQL Server no PHP?']
X_predict = tfidf.fit_transform(predict_text)
br_prediction = binary_rel_clf.predict(X_predict)
print(br_prediction)

However, I got this error:

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 56 is different from 4)

What "dimension" do I need to change to run predict() correctly?

score 1 · Accepted Answer · answered Jul 24 '21 at 22:14

You are using a TfidfVectorizer to transform your text features. You should fit the transformer only once on the training data, which is corpus in your case. When preparing the data to test/predict, you should however use the transform method and not fit_transform again, since that would refit the transformer.

Change the following to make it work:

X_predict = tfidf.transform(predict_text)

Matmul error when trying to predict a new text using skmultilearn.BinaryRelevance

1 Answers1