3

Suppose I had a list of slogans (short, multi-word phrases), and people had voted for the ones they liked best, and I wanted to assess which words, if any, made some slogans more popular than others. What would be the best way to achieve this? My first thought was just to find all the unique words in the set of slogans and score each one as the average number of votes of all the slogans that contain said word, but frequency should also come into play in some fashion, I think, so that the following should be true:

  • If Word A occurs in only the slogan that got the most votes and Word B only occurs in the slogan that got the second-most, Word A is more "popularity-generating"
  • However, if Word A occurs only in the top-ranked slogan and Word B occurs in both the second- and third-ranked slogans, Word B should win, since it pushed more slogans to the top.
  • However, a single occurrence of Word A in the top slogan should still trump three appearances of Word B in other slogans if they're, say, in the middle, or bottom half, of the pack (that is to say, there needs to be a balance of vote-getting and frequency in scoring).

I also want to eliminate words that are generally common (e.g., "the" or "of"). This is sort of related to questions about identifying trending words that have been asked in the past, but different because change over time isn't a factor. I'd be happy just to be pointed in the right direction about this as far as literature is concerned, but I'm not really sure what to look for. Is this a class of problem that other people deal with?

Andrew Pendleton
  • 761
  • 5
  • 13

3 Answers3

2

This is a machine learning question. You are trying to learn a model from supervised data. To do this, you could run a simple algorithm that's like Perceptron or SampleRank (pdf):

First you define features that apply to the words in a slogan. Features can be shared across words, e.g. features of the word "peace" could be:

  • "peace",
  • "noun",
  • "abstract-noun",
  • "short-noun",
  • "starts with p",
  • "ends in 's'-sound",
  • ...

The first feature "peace" is a unique feature that fires only on "peace", whereas the other features can also fire on other words.

Each feature has a weight (higher is better). So you have a feature vector and a weight vector. This will enable you to assign a weight (score) to any slogan (just the sum of all weighted features that fire on the words in the slogan). All weights are initialized to 0.0.

Now you start training:

You loop over all pairs of slogans. For each pair you know the true ranking (according to the votes you already have). Then you compute the ranking according to the features and their current weights. If the true ranking and the ranking according to your current feature weights (i.e., according to your current model) is the same you just move on to the next pair. If your model assigned the wrong ranking you correct the feature weights: You add 1.0 to the weights of the features that fire on the better slogan (the one that's better according to the people's vote) and subtract 1.0 from the weights of the features that fire on the worse slogan (its score was obviously too high, so you're lowering it now). These weight updates will affect the scores that your model assigns to the next pairs, and so on.

You run this loop several times, until your model got most of the pairs right (or some other convergence criterion).

Typically, you don't really add or subtract 1.0, but eta times 1.0, where eta is the learning rate, which you can set experimentally. Typically it is higher at the beginning of training and is gradually lowered during training, as your weights are moving in the right direction. (See also stochastic gradient descent.) To get started, you could just set it to 0.1 as a constant.

This procedure takes care of the stop words ("the", "of", ...) as they should occur equally often in good and in bad slogans (and if they really don't then you learn that too).

After training, you can compute the score for each word according to the learned feature weights.

Frank
  • 64,140
  • 93
  • 237
  • 324
0

How about Bayesian inference?

SRM
  • 1,377
  • 1
  • 10
  • 17
  • My impression is that Bayesian inference is mostly about particular pieces of evidence either confirming or not confirming a hypothesis. My question, here, isn't particularly yes-or-no, which is to say, it's not so much "is this word popularity-promoting or not?" but rather, "what's the ranking of the extent to which these words promote popularity?" Do you still think Bayesian analysis would be applicable in this context? – Andrew Pendleton Dec 03 '10 at 22:41
  • Hmm, now it sounds like you are looking at something like PageRank that Google uses (or used to use?). It basically makes a graph of terms connected by relevancy. In essence, the terms with the most connections are relevant. I agree Bayesian wouldn't give you the relevance of the terms in this case. You could use it to infer relationships you don't know about, though. – SRM Dec 03 '10 at 23:29
0

I think I would use an algorithm that does this things:

  1. Eliminates stop words (common words) (http://en.wikipedia.org/wiki/Stop_word)
  2. Normalize and stem words (http://en.wikipedia.org/wiki/Stemming)
  3. Calculate normalized word frequencies and votes
  4. Order normalized words by votes / frequency
Nicolas Bottarini
  • 239
  • 1
  • 4
  • This is close, I think. The stop word elimination is a bit tricky, because (and I should have been more clear, here) it's probably not just stop words that I want to eliminate, but more likely any words that don't occur any more frequently in this dataset than they do in general. That's just a hunch, but I suspect that the results will be more interesting if I focus on words that are statistically improbable. Maybe I could compare this dataset to some reference corpus, and otherwise follow your approach? – Andrew Pendleton Dec 03 '10 at 22:44
  • If you have a corpus that reflects the topic of your phrases or is general and big enough i think it will work. You'll have experiment a bit and see the results. – Nicolas Bottarini Dec 03 '10 at 22:51