Mapping the Contents in Wikipedia

Having just returned from CHI2009 conference on Human-Computer Interaction, many of the topics there focused on where and how people obtain their information, and how they make sense of it all. A recent research topic in our group is understanding how people are using Wikipedia for their information needs. One question that had constantly come up in our discussion around Wikipedia is what is exactly in it. We have so far done most of our analyses around edit patterns, but not so much analysis have gone into what do people write about? What topics are the most well-represented? Where topic areas have the most conflict?

In one of our recent CHI2009 papers, we explored this issue. Turns out that Wikipedia have these things called Categories, which people use to organize the content into a pseudo-hierarchy of topics. We devised a simple path-based algorithm for assigning articles to large top-level categories in an attempt to understand what topic areas are the most well-represented. The top level categories are:



Using our algorithm, the page "Albert Einstein" can be assigned to these top-level categories:


This mapping makes some intuitive sense. You can see that the impact Albert Einstein has made in various areas of our society such as science, philosophy, history, and religion. Using the same ideas and algorithm, we can now do this mapping for all of the pages in Wikipedia, and find out what top level categories have received the most representation. In other words, we can figure out the coverage of topic areas in Wikipedia.


(You may have to click on the graphic here to see it in more detail.)

We can see that the highest coverage has gone toward the top-level category of "culture and the arts" at 30%, followed by "people" 15%, "geography" 14%, "society and social science" 12%, and history at 11%. What's perhaps more interesting is understanding which ones of these categories have generated the most conflicts! We used the previously developed concept called Conflict Revision Count (CRC) in our CHI2007 paper, and showed which top level categories have the most conflicts:



In this figure, the categories are listed in order of the total amount of conflicts clockwise from "People". This means that People did receive the most amount of conflict, followed by Society and Social Sciences, etc. However, the percentages in each topic is normalized by the number of article-assignments in that topic. So the metric developed here can be interpreted as the amount of conflict in each topic that has been normalized by the size of the topic, which can be interpreted as the amount of contentious in articles of the topic.

"Religion" and "Philosophy" stand out as highly contentious despite having relatively few articles. Turns out that "philosophy" and "religion" have generated 28% of the conflicts contentious-ness each. This is despite the fact that they were only 1% and 2%, respectively, of the total distribution of topics as shown above.

Digging into religion more closely, we see that "Atheism" have generated the most conflict, followed by "Prem Rawat" -- the controversial Guru and religious leader, "Islam" and "Falun Gong".



Wikipedia is the 8th ranked website in the world, so it is clear that a lot of people get their information from Wikipedia. The surprising thing about Wikipedia is that it succeeded at all. Common sense would suggest that an encyclopedia in which anyone can edit anything they want would result in utter nonsense. What happened is exactly the opposite: Many users and groups have gotten together to make sense of complex topics and debate with each other about what information is the most relevant and interesting to be included. This helps with us keeping sane in this information world, because we now have a cheap and always accessible content on some of the most obscure content you might be interested in. At lunch today, we were all just wondering what countries have the lowest birth rate. Well, surprise!! Of course, there is a page for that, which we found using our iPhones.

The techniques we have developed here enable us to understand what content is available in Wikipedia and how various top level categories are covered, as well as the amount of controversy in each category.

There are of course many risks in using online content. However, we have been researching tools that might alleviate these concerns. For example, WikiDashboard is a tool that visualizes the social dynamics behind how an wiki article came into its current state. It shows the top editors of any Wikipedia page, and how much they have edited. It also can show the top articles that a user is interested in.

We are considering adding this capability to WikiDashboard, and would welcome your comments on the analysis and ideas here.

All web users can guide the content in Wikipedia by participating in it. If we realized that the existence of our society depends on the healthy discourse between different segments of the population, then we will see it not just as a source of conflict, but a source of healthy discussion that needs to occur in our world. By having these discussions in the open (with full social transparency), we can ensure all points of view are represented in this shared resource. Our responsibility is to ensure that the discussion and conflicts are healthy and productive.



Reference:
Kittur, A., Chi, E. H., and Suh, B. 2009. What's in Wikipedia?: Mapping Topics and Conflict using Socially Annotated Category Structure. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 1509-1512.

Differences between Social Tagging and Collaborative Tagging

I'm here at the InfoVis conference in Sacramento and a conversation with Marti Hearst over at UCBerkeley just reminded me why I have been bothered by the 'confusion' between the phrases "social tagging" and "collaborative tagging" for quite some time. In fact, Wikipedia has a redirection of "Social Tagging" to "Collaborative Tagging" (see http://en.wikipedia.org/w/index.php?title=Social_tagging&redirect=no). This, I would argue, is wrong. Why?

'Collaborate', according to the American Heritage Dictionary, is "to work together, especially in a joint intellectual effort." The problem is that tagging features in many of the popular Web2.0 tools such as Flickr and YouTube are not really 'collaborative', since users aren't really working together per se. In YouTube, for example, only the uploader of the original video clip can specify and edit the tags for an video. Most of the time, in Flickr, one only tag their own photos. However, Flickr is somewhat more collaborative than YouTube because the default setting for any account is to allow contacts such as friends and families to also tag the photos.

Both of these two systems don't seem that 'collaborative', because, to me, collaboration implies shared artifact, shared workspace, and shared work. On the other hand, 'social' is "living or disposed to live in companionship with others or in a community, rather than in isolation". In other words, simply existing and having some relation to others in a community. So for example, I would argue that in YouTube, we have social tagging but not collaborative tagging, because while users tag their uploaded videos in the context of a online social community, and they do not collaborate to converge on a set of tags appropriate for that video.

The use of the term 'collaborative' in past Computer-Supported Cooperative Work (CSCW) field has especially come to imply
a shared workspace. With shared workspaces, often there are some elements of coordination and conflicts involved as well (and hopefully conflict resolution as well). So in contrast to YouTube, the most 'collaborative' tagging system I know is the category tagging system in Wikipedia. Anyone can edit the category tags for an article. They can remove, add, discuss, and revert the use of any tag. In this case, the category tags are shared artifacts that anyone can edit inside a shared workspace. The work of tagging all 2 Million+ articles in Wikipedia is shared work among the community.

It's perhaps interesting to note that somewhere in between YouTube and Wikipedia tagging is perhaps the bookmarking system del.icio.us. In del.icio.us, there is a shared artifact (the tagged sites or URLs), and there is shared work of tagging all of the websites and pages out there on the Web. However, there is less of a notion of a shared workspace. My tags for an URL could be and probably is different from someone else's tags for the same URL. I also have the capability of searching within just my own del.icio.us space. So from least collaborative to the most collaborative, we have YouTube, then del.icio.us, and then finally the category tagging system in Wikipedia.

A simple way to explain this is that one must be social in order to collaborate, but one need not be collaborative to be social. So in summary, I would argue that social tagging is a superset of collaborative tagging. But a social tagging system may not necessarily be a collaborative tagging system. We should change the definitions in Wikipedia to distinguish between these two types of systems.

Talk video: Enhancing the Social Web through Augmented Social Cognition research

PARC Forum: May 1, 2008, 4:00 p.m., George E. Pake Auditorium, Palo Alto, CA ,USA

Enhancing the Social Web through Augmented Social Cognition research

Ed Chi, PARC Augmented Social Cognition group

We are experiencing the new Social Web, where people share, communicate, commiserate, and conflict with each other. As evidenced by Wikipedia and del.icio.us, Web 2.0 environments are turning people into social information foragers and sharers. Users interact to resolve conflicts and jointly make sense of topic areas from "Obama vs. Clinton" to "Islam."

PARC's Augmented Social Cognition researchers -- who come from cognitive psychology, computer science, HCI, sociology, and other disciplines -- focus on understanding how to "enhance a group of people's ability to remember, think, and reason". Through Web 2.0 systems like social tagging, blogs, Wikis, and more, we can finally study, in detail, these types of enhancements on a very large scale.

In this Forum, we summarize recent PARC work and early findings on: (1) how conflict and coordination have played out in Wikipedia, and how social transparency might affect reader trust; (2) how decreasing interaction costs might change participation in social tagging systems; and (3) how computation can help organize user-generated content and
metadata.


Bio:
Ed H. Chi is a senior research scientist and area manager of PARC's Augmented Social Cognition group. His previous work includes understanding Information Scent (how users navigate and make sense of information environments like the Web), as well as developing information visualizations such as the "Spreadsheet for Visualization" (which allows users to explore data through a spreadsheet metaphor where each cell holds an entire data set with a full-fledged visualization). He has also worked on computational molecular biology, ubiquitous computing systems, and recommendation and personalized search engines. Ed has over 19 patents and has been conducting research on user interface software systems since 1993. He has been quoted in the Economist, Time Magazine, LA Times, Slate, and the Associated Press. Ed completed his B.S., M.S., and Ph.D. degrees from the University of Minnesota between 1992 and 1999. In his spare time, he is an avid Taekwondo black belt, photographer, and snowboarder.

Visualization used to improve team coordination

This past Thursday I spent some time at IBM Almaden research center to attend the NPUC conference, which focused on the future of software development. In computing, software development is one of the most energy intensive collaborations, and often requiring significant coordination. There are elements of competition thrown-in for good measure, and of course, everyone is working in the same workspace, which is often coordinated by version control software. Sounds quite like Wikipedia, doesn't it?

One of the interesting talks given at NPUC was Gina Venolia's talk on using visualizations to represent the structure of the code. This representation can be used individually to make sense of the system, as well as being used by a team to explain structure to others. As a map to the system, they help anchor conversations between developers by providing for an intermediate representation of the knowledge structure that they must share for effective coordination.

This is a fascinating area to think about how augmented social cognition ideas could provide for better tools for collaborative software development. For example:

* Each developer could get an color on the map. Overlaps between two developer can then be easily visualized to see areas where they need to coordinate in the past and in the future.

* Building up an understanding from the code of who is working with whom, annotations and comments made by one developer could be send over to another developer's map when code is checked into the system.

* Social analytic can be used to discover where developers are clashing with each other (like how we have discovered conflicts in Wikipedia).

Microsoft Research indeed has been thinking about awareness tools toward this direction. A project named FASTDash works to increase awareness between developers in software teams. Lots of exciting possibilities!

Conflict and Power Structure in Wikipedia

We presented two papers at the CHI2007 conference. One paper was on the conflicts and coordination costs of Wikipedia. (Paper here.)

The other paper was an alt.chi paper on the power structure of Wikipedia. (Paper here and here.)

The room was absolutely packed (easily 200+ people there), and they were spilling out into the hallways! Picture above was found on flickr.