-1

I'd like to do a k means clustering with book text data that has 'title', 'genre', 'review', and 'synopsis' columns.

I want to use the 'title' as the indicator, or primary key, for clustering, but I'm not sure how to use multiple columns for this.

I know that I first have to vectorize the data, but vectorization takes in series data and not dataframe values; so here, again, I don't know how to use all the columns as I want to.

Jason Kim
  • 3
  • 3

1 Answers1

0

You can vectorize each column separately and concatenate the results.

Just make sure you do a sparse concatenation.

However, clustering text with k-means is not at all working well. K-means is very sensitive to outliers and noise, and test is full of noise. Fundamental assumptions of k-means (k signals, and i.i.d. Gaussian error) do not hold for text. Good luck...

Has QUIT--Anony-Mousse
  • 76,138
  • 12
  • 138
  • 194
  • Can you provide a link or reference to a working example if any? Examples in which text classification has been done on multiple columns? Something that would have the basic rundown of what all I need to do in order to get a working clustering algorithm for text data with multiple columns? – shinz4u Sep 23 '18 at 06:14