Cliff Lampe and Paul Resnick have a paper out called Slash(dot) and Burn: Distributed Moderation in a Large Online Conversation Space
(note colon, for extra academic juju). Although much of it is background for people not familiar with slashdot's system, the section on design implications is quite interesting
Slashdots design, and the usage patterns that have emerged, highlight tensions among four design goals for distributed moderation systems. First, comments should be moderated quickly. Second, they should be moderated accurately according to the community norms. Third, each individual moderator should have limited impact on any particular comment. Fourth, the burden on moderators should be minimized, to encourage their continued participation.
Consider the tension among timeliness, accuracy, and minimizing the influence of individual moderators. In the Slashdot system, two to five people (depending on a comments initial score) must provide positive moderations before a comment reaches a score of +4. This limits the impact of any individual moderator. But more than 40% of comments that reached +4 took longer than three hours to reach it; in three hours, the typical conversation was already half over. An alternative design would give more weight to early moderators, which would lead to earlier identification of treasures (and trash) but would give more power to those early moderators and lead to more errors caused by items having inappropriately high or low scores that would have to be corrected by future moderators.
And, over on Kuro5hin, there is an old post (Oct 2003) by localroger, Notes Towards a Moderation Economy
, which notes the same aspects of the slashdot moderation system, and goes on to propose a long list of alternative techniques:
An economy, like an ecosystem, is more stable if it has multiple feedback pathways. Any single feedback pathway is prone to catastrophe; the cockroaches eat all the bamboo and subsequently starve, or the one company with all the money fires all its own employees to save money, but they're also its customers and as a result the economy goes bankrupt. But if there are many species with interlocking relationships, or many participants in the economy, a catastrophic turn in any single path does not ruin the system.
With this in mind, let's consider some additional reward systems that could be automated in an electronic community:
Equity should be worth something, so that one has an incentive to use a single account and keep it in good standing. [...]
Leaving highly rated posts should grow one's equity, and leaving poorly rated posts should shrink it.
Rating itself should cost some equity, so that one thinks before doing it.
Extreme rating might cost extra, so that it is meaningful. For example, the old 5-point rating system was eliminated because most people rated 1 or 5; this could be fixed as follows:
This was so awful I was willing to spend 2 points kicking it.
This was pretty bad. Cost me a point to say so.
Read it, nothing special. Costs nothing to say so.
This was pretty good, worth a point to say so.
This was so good I was willing to spend 2 points saying so.
The comments on the Kur05hin piece are also well worth reading (paging Tom Coates...)