(Meta: not sure whether this question is suitable for StackOverflow, it has an AI/statistics slant. Feel free to recommend which StackExchange site is more suitable.)
A lot of reputation systems, like eBay, Slashdot, AirBnB and StackOverflow base a user's reputation on how many positive and negative reviews someone's effort receives. In these cases, users vote on others based with a Good/Bad signal, like vote up or down. In most cases, this is an absolute rating: an answer is helpful or it's not, or a comment is good or not, a seller/buyer is good to deal with or not, etc.
I want to build a reputation system where each individual rating is inherently based on how good the person being rated is in relation to the person doing the rating (just because that's the domain). I want to be able to recommend something in an 'absolute' way, even though rating is unavoidably relative. Recommending something to a person that is too distant in rating (either too good, or too bad) negatively affects their experience. The goal is to recommend like-for-like.
Consider this made up scenario.
A system exists to connect people who want to play poker together. Within a poker game, players are competing with each other, and it would negatively affect everyone's enjoyment of the game if it was either too easy, or too hard. Players are rated based on their skill, relative to the person rating (i.e. 'she is better than me' is an upvote, 'he is worse than me' is a downvote). Through all these ratings, the system should be able to rate a player out of 5, so that someone looking for other players can find people at their own level.
I suspect that dating services might do something like this, based on people's attractiveness. That sounds horrible, I know, but I guess setting two people up who are far apart in that regard is probably a waste of both party's time, due to how superficial us humans can be.
Are there reputation systems designed to "rate on a curve", such that a user will be able to find 'their point' on the scale, and be recommended something close to that point? How could a system like that be programmed?
