Monday, March 8, 2010

Wikipedia's People-Ware Problem

Last week, we hosted a visit from the Wikimedia Foundation on issues relating to our work on community analytics, and what it tells us about Wikipedia's problems and possible solutions. Naoko Komura (pictured at right) of the Wikimedia Usability Initiative, as well as Eric Zachte, the staff data analyst (also pictured at right), spoke very eloquently about how we can create social tools to direct the best social attentions to the needed parts of Wikipedia.

Fundamentally, Wikipedia has always had a "people-ware" problem: the distribution of the expertise that is freely donated to the right places.  It has been and always will remain its greatest challenge. The amazing thing about Wikipedia is that it managed to do this for so long, such that a valuable knowledge repository can be built up as a result.  At first, people simply came because it was the place to be.  Now, we have to work a little harder.

We spent a lot of time talking about the best way to model this people-ware problem, either using biological metaphors (evolutionary systems with various forces), or economic models (see last post here).  However, one thing to be aware of is the danger of "analysis paralysis", where you spend so much time analyzing the problem, and forget that there are already many ideas that have been generated for moving the great experiment forward.

For example, there are many places in Wikipedia that are not well populated. It's well-known that many scientific and math concept articles, for example, could use an expert-eye to catch the errors and explain the concepts better. How can we build an expertise finder that would actually invite people to fix problems that we know exists in Wikipedia?

Another idea might be to have the whole system be more social. Chris Grams blogs about a part of this idea here. We suggested some time ago to have a system like WikiDashboard, where you actually show the readers what the social dynamics have been for a particular article.

Wikipedia was created in 2001, when social web was still in its infancy. During the ensuing 9 years, it has changed very little, and I would argue Wikipedia have not kept up with the times. Lots of "Social Web" systems and new cultural norms have been built up already.  For example, I suspect that many of us would not mind at all to reveal our identities on Wikipedia, and we might like to login with our OpenIDs and even have verified email addresses so that the system can send me verification/clarification/notification messages. The system perhaps should connect with Facebook, so that my activities (editing an article on "Windburn") is automatically sent to my stream there. My friends, upon seeing that I have been editing that article, might even join in.

I think that Wikipedia is about to change, and it is going to become a much more socially-aware place. I certainly hope that they will tackle the People-Ware (instead of the Tool-Ware) problems, and we will see it become an exciting place again.


Anonymous said...

I think Wikipedia worked best before the media started encouraging it, and it became well known.

In those days you contributed because it was a great resource.

Then we got the WikiMafia, who failed to understand the rule "There are no rules". Then... Now we have a fair share of inane idiots putting "citation required" over, and all sorts of other annoying messages. These folk should just fix it not cop out, be lazy and spoil the pages. Then you have the WikiMafia who haven't got a clue and edit anyway, unbelievable.

In part that was induced by "being the place to be", fatal, you get people who want to be fashionable. Social media ideas will maybe kill it off with an influx of people who are unsuited.

I think trying to be facebookish is a retrograde move. Banning anonymous posting might be an improvement, there's a lot of mischief in there hidden behind unknown people and sock puppets.

Some parts of Wikipedia are still very, very good, others...

Anonymous said...

never heard the term "WikiMafia" that Mike Gale is using but I definitely understand it, having it experienced it few times.

I suppose until now stigmergy and auto-organisation were too costly to research and implement more seriously but either Wikipedia leverage such studies and projects like or, like so often seen in epistemology, business but first and foremost in biology, a new alternative built upon it will pursue the original ideal.

James Salsman said...

The most substantial "people-ware" problem facing Wikipedia today, as far as I can tell, is active administrator attrition. Who has ideas for addressing that? I've been thinking about it for months without much success.

Ed H. Chi said...

@Mike Gale: A healthy community, I think, needs all kinds of people, including administrators, influx of newbies, domain experts. I would contend that perhaps, yes, even the "WikiMafia" might serve an important role in shaping the community. The problem is having the right mixture of these people. The current analysis suggests a bias against newbies, which has stagnated the growth of Wikipedia.

@Utopiah: Given a set of people who are willing to work on Wikipedia, how would you propose a self-organizational system that would make them as efficient as possible? Do you not think of the current system to be self-organizing? I think of it as at least a version of it: micro-coordination behaviors eventually arise to be macro-coordination policies. Ironically, the over-arching policies often end up discouraging the smaller-scale micro-coordination behaviors.

@Salsman: Thanks for the new data on admin attrition. I think the phenomenon arises as a result of lack of admin replacements. My hypothesis is that in the past, when some admins have left, there were others who took their place because we had a healthy flow of newbies who eventually get involved enough to be voted in. Now that flow has been cut off because we have less number of newbies joining every month.

Gregory Kohs said...

Ed, is it a scientific process if you have an unwavering viewpoint that the subject matter (in this case, Wikipedia) is inherently a wonderful phenomenon and couldn't possibly be wrongly governed or nefariously exploited?

My viewpoint is that you are studying something like malaria, but you (as the scientist) have chosen an agenda that takes the side of the eukaryotic Plasmodium protist. What a tragedy if this beautiful parasite were to become extinct, you seem to be saying.

Ed H. Chi said...

@Kohs: Scientists start with evidences and tries to draw conclusion from it. The evidences seem to suggest thusfar that many people obtain good information from Wikipedia. Just as Google and other sites have become an inseparable part of the Web fabric, Wikipedia seems to have done the same. For simple verification, witness the amount of traffic that goes to Wikipedia, and the amount Google seems to return Wikipedia results in its searches.

However, I don't believe I said that it is 'inherently' a wonderful thing. I said that it is (present tense) a wonderful thing for knowledge. I believe the evidence suggest that the folks running it (the foundation, and some elite Wikipedians) are trying to govern it as best as they could. Unfortunately, some do nefariously exploit it.

I'm interested in documenting the malaria, the way in which it is wrongly governed as well as the way in which it is "correctly" governed (if such practice can exist). I'm also interested in documenting the nefarious exploits, and think of ways to avoid it.

I think you can see how our work has both documented the good and bad parts of the social dynamics within Wikipedia. The recent discovery of how low-edit-count-editors have different revert experiences than high-edit-count-editors is a good example.

But yes, I suppose I stand accused of trying to advance the state of the art. It is by far one of the most interesting social system to study, so as a scientist, not to study the "malaria" seem like the real crime to me.

James Salsman said...

A better metaphor than malaria might be: a new symbiotic organism is discovered, but the extent to which it's commensal or parasitic is not entirely clear, and may even be varying over time. So let's study it carefully so we have the ability to take corrective measures if aspects of it are parasitic and try to nurture the aspects which seem to be commensal.

Ed H. Chi said...

@James: I'm not dogmatic about what metaphor we use to describe it. But here is one: If we're medical doctors, and Wikipedia is a patient, I don't want to make the declaration that it is dead when in fact it isn't. I want to help the patient to stay healthy if it is already healthy, and I want the patient to be healthy if it is sick.

@Kohs seems to be saying that it is very sick. Well, I would like to find evidences of that, if that's the case. However successful/failed it is, our research is focused on two things:

(1) Can we characterize it? That is, can we understand what is happening, and can we find metrics that tell us anything? Are our metrics good metrics? Is there truly an aggregation of lots of opinions or it is run by elites? We want to be data-driven, uncolored by any lenses, as much as possible.

(2) Can we make it better? That is, regardless whether it is a bad or good thing, can we improve it? Can we make it more efficient? Reflect more POV? Encourage expert contribution? Not waste people's time? Reduce errors? Improve demographics? Reduce conflicts? Better coordination?

ThinkerLess said...

In "Slowing growth from Wikipedia", it is found that "a greater proportion of the overall edits is being devoted to overhead activities such as coordination, policy setting, and governance": doesn´t this represent growth. The resulting product is different (it is not articles), but it is a contribution to the improvement of the coordination mechanisms, and hence, an attempt to solve the "people-ware problem". Don´t you think so?

I posted in relation to that,

BTW, congrat for the paper, make more like it.