In our research on Wikipedia, we have been using a broad framework published as an one-page article in CACM (Communication of the ACM) in 2005 by Denning et al. on the perceived risks in using Wikipedia contents. The question before researchers is how to mitigate these risks while enabling a vibrant social community who wants to get together to build a encyclopedia and help each other obtain knowledge. As Denning's article mentions: "But will this process actually yield a reliable, authoritative reference encompassing the entire range of human knowledge?"
In the framework in thinking about answers to this question, Denning's article suggests that numerous risks that we should consider:
- Accuracy: how can you be sure that the information in the article was actually accurate and not some misrepresentation of the fact?
- Motives: how can you be sure of the motive of the editors were to present the facts and only the facts, and not opinions? For example, as we have discussed before, how can we be sure that editors to political candidate pages are not there to simply push their political agenda? For example, what does User:Jasper23's editing history tell you about his potential political positions?
- Uncertain Expertise: How do we determine the expertise levels of the people who are editing in Wikipedia? Some appears to really know what they're talking--for example, User:BillCl was the top editor of the NASA page, and appears to have done a bunch of edits around aviation related topics. On the other hand, the top editor (Wasted Time R) of the Hillary Clinton page appears to be also to be big fans of Bruce Springsteen, Elton John, Dixie Chicks, Tony_Bennett, and other musicians. So it is less clear this user is an expert on political positions of Hillary Clinton as compared to other candidates.
- Volatility: If the topic was just in the middle of a huge debate, then the content of the article could be really unsettled.
- Coverage: Coverage of certain topics appears to be better in Wikipedia than others. What are the inclusion and exclusion standards is less than clear.
- Sources: Many articles do not cite authoritative sources, so it is hard to trace and find out if the information is actually accurate.
The article then goes on to say that "[WP] cannot attain the status of a true encyclopedia without more formal content-inclusion and expert review procedures." Our WikiDashboard tool is precisely designed to help with collaborative review of Wikipedia editing history and patterns.
