9

This question is more related to logic than any programming language. If the question is not apt for the forum please do let me know and I will delete this.

I have to write a logic to calculate scores for blogs for a Blog Award website. A blog may be nominated for multiple award categories and is peer-reviewed or rated by a Jury on a -1 to 5 scale (-1 to indicate a blog they utterly dislike). Now, a blog can be rated by one or more Jurors. One criterion while calculating final score for a blog is that if a blog is rated positively by more people it should get more weightage (and vice-versa). Similarly a blog rated -1 even by one Juror should have its score affected (-1 is sort of a Veto here). Lastly, I also want to have an additional score based on the Technorati rank of the blog (so that the final score is based on a mix of Juror rating + Technorati ranking).

Example: A blog is rated in category A by total 6 Jurors. 2 rate it at 3, 3 rate it at 2 and 1 rate it at 4. (I used to calculate the score as (2*3 + 3*2 + 1*4)/6 = 16/6 = 2.67 to get weighted average but I am not satisfied with this, primarily because it doesn't work well when a Juror rating is -1. Moreover, I need to add the Technorati ranking ranking criteria too) .

Could you help me decide the best way to calculate the final scores (keeping the rating method same as above as that cannot be changed now)?

Dchucks
  • 1,189
  • 5
  • 22
  • 48
  • This question seems fine for SO. You're asking for the 'best' algorithm to assign scores. – paxdiablo Jun 15 '09 at 07:15
  • why wouldn't you just use an `if (scores include -1) { remove from competition }` before the scoring? That's more or less what a veto means, right? – naught101 Jul 11 '12 at 05:30

4 Answers4

5

If you want to weight the effect of a -1 rating more strongly, use the same average score calculation but substitute -10 whenever you see -1. You can choose a value other than -10 if you don't want a negative rating to weight as strongly.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • Thanks Greg. What do you suggest about the Technorati ranking? And are you satisfied the way weighted average was being calculated? – Dchucks Jun 15 '09 at 07:49
  • You haven't specified what the Technorati ranking is. Is it also a value between -1 and 5? Or is it something else? How strongly do you want to weight it against the rankings of your jurors? These are the kinds of questions you will need to answer to determine how to use that ranking too. – Greg Hewgill Jun 15 '09 at 08:23
  • Greg, Technorati ranking is a score given by Technorati.com to a website based on various factors such as traffic, Linkback etc. Lower the rank value, better is the site. So a site with Technorati ranking 3000 is better off than one with a ranking 40000. – Dchucks Jun 16 '09 at 05:22
  • What is the range of Technorati values? It's also depends on how important you think the Technorati rating is. I mean you could just split how much each score matters, make the jurors account for some percentage of the score say 80% and then the Technorati rating could be the other 20%. But I don't really know how important you feel it should be. Also if you want to have more good scores count for more, just shift the score up based on the average score. – JSchlather Jun 16 '09 at 20:48
4

You might look at using the lower bound of the Wilson score interval for your ratings.

See http://www.evanmiller.org/how-not-to-sort-by-average-rating.html for more details. Although, there, it is used for the simpler Bernoulli case.

The gist is if you have a lot of ratings you have a higher degree of confidence in your scoring. You can then combine the scores from your local ratings and the Technorati ratings, by weighting the scores by the number of voters locally and on Technorati.

As for wanting a single -1 vote to have high impact, just remap it to a large negative value proportional to your desired impact before feeding it into your scoring formula.

Edward Kmett
  • 29,632
  • 7
  • 85
  • 107
4

Calculating a score based on votes will be pretty easy. Adding the technorati rank will be the tricky part.

I made a quick script that calculates some scores based on this algorithm

score = ( vote_sum - ( vetos * veto_weight ) ) / number_of_votes

you can change the url paramters to get different values

There are a lot of ties, so maybe you could use technorati blog rank as a tie breaker

Galen
  • 29,976
  • 9
  • 71
  • 89
0

you could internally work with scores from 0 to 6. Just do a shift by one, calculate the score and shift back. I guess the -1 has some disrupting effekt on your calculation.

Tobias Langner
  • 10,634
  • 6
  • 46
  • 76