Loading...
 

Automatic Reputation Management

Context

This practice is useful for ensuring quality in any Collaborative translation initiative, but it is particularly relevant where you are tapping into a large crowd of unknown contributors.

Problem description

Ensuring quality in Collaborative translation can be a challenge. One way to approach this issue is to vet contributors, instead of individual contributions. In other words, make sure that only good contributors are allowed to create or modify content on your site.

But how can you tell who the good contributors are?

Solution

One way to approach this is to vet contributors after the fact, by keeping an eye on their contributions, and seeing what the rest of the community thinks about their quality.

This is usually done by allowing other members of the community to Peer Review contributions made by one individual, and by having the system produce some sort of aggregate metric which summarize the community's overall assessment of that individual.

The reputation can be expressed numerically (ex: scale of 1 to 10), or more qualitatively (ex: a list of testimonials by other members of the community).

In the case of numerically expressed reputation, it is common for the Peer Review to have a reciprocal effect. For example, if Jane is a translator who has a very high reputation among a community, and Joe rates one of her translation very negatively, then this may reflect more negatively on Joe's reputation as a reviser, than it would reflect negatively on Jane's reputation as a translator.

Links to related patterns

  • Assumes that the system supports some form of Peer Review.
  • Entry Exam is another way of differentiating between good contributors and bad ones. The difference is that there, contributors are screened beforehand, whereas Automatic Reputation Management screens them after the fact.
  • Like Point System, a reputation management system can be used to publicly recognize the best contributors (for example, with the Contributor of the Month pattern). The main difference is that Point System tends to emphasize the quantity of work, whereas reputation tends to emphasize quality.

Real-life examples

  • Translated.net is an example of a community that has implemented numeric reputation management.
  • Proz is an example of a community that uses a more qualitative reputation management.