Wikipedia: Increasing Reliability
Numerous critics (like Andrew Orlowski) of Wikipedia have come out swinging since the news broke not long ago of various abuses its articles have suffered at the hands of ignorant and malicious “contributors”. I feel that the essential problem with Wikipedia (a site which I use frequently and value highly) is not so much the quality of the content as that unsophisticated users are in the dark about how to judge the quality of any particular article. The inability to assess the quality results in a loss of reliability.
While improving quality is a desirable goal, there is no known means for doing so short of improving the abilities of contributors, and/or increasing the number of contributors who are skilled and knowledgeable. Current contributors can gain experience, and new contributors can be sought, but at any one point in time, the problem of insufficient resources will always exist, and the quality of articles will always be in question.
As such, the answer to the problem of Wikipedia’s reliability (which has been wrongly confused with its moral responsibility), is to improve the means for evaluating a page’s content, and to make this evaluation transparent and independently verifiable.
My proposal is quite simple. Pages should be publicly evaluated. Specifically, they should be vetted and ranked. Ranking can be done both by readers and editors. Readers could rank pages voluntarily, but editors should be required to vet a certain number of pages. In both cases, ranking simply means judging a page’s quality on various criteria versus an idealized standard. Vetting should be done by experts, who would then fulfill that role for a page (or, ideally, a subject area and all enclosed pages) for some period of time. It would be most desirable for expert editors to commit to reviewing a page periodically, possibly based in part on frequency of edits, but as with all voluntary work, the commitment of a vetting expert will vary. Experts would be required to have user pages with up-to-date biographies which explained the source(s) of their expertise.
Some pages could be assigned multiple experts, but how to resolve disagreements is unclear. I would argue to allow a maximum of two experts, and the results of their assessments would not be integrated (unlike ranking, which is a voting process which implicitly integrates opinions). Choosing an expert could easily become a source of disagreement, but many possible organizational techniques to overcome this can be found.
The work of editors automatically involves vetting to some degree, but the process is not sufficiently explicit. There is no equivalent to ranking of which I am aware. In both cases, ranks and expert evaluations should be visible on the page with the article’s content, probably on the side bar. Evaluations should be dated for easy comparison to the date of last (major) edit of the article, which probably should also be posted clearly on the article’s content page (rather than only on the history page, as it is now).
Rankings should also account for changes to the article over time, so every vote should be dated so that it can be associated with a particular version of that article. Algorithms to do this would have to be agreed upon.
Finally, articles can be evaluated automatically for churn, the rate at which their content changes meaningfully, where a meaningful change is not a minor change. Minor changes are identified as such by editors, but can usually be identified automatically, too.
If Wikipedia pages are transparently evaluated in this manner, any reader will be able to quickly get a sense of the reliability of any given article, simply by looking at current rankings. As articles stabilize, their rankings will become more dependable. Expert evaluations can be themselves judged for applicability by both the time they were performed, and the reader’s opinion of the expert.