Thursday, October 16, 2008

A new live-data version of WikiDashboard for Wikipedia

The ASC group (and Bongwon Suh in particular) is pleased to announce a new version of WikiDashboard for Wikipedia. In this new version, we have:

* Live Information!
WikiDashboard now uses the live feed of English Wikipedia powered by MediaWiki Toolserver. The dashboard will show any changes made on each page almost instantly. Note that the earlier version has been showing information as of April 2008. For example, you can see who's been active in pages such as: Sarah Palin's page or the US President Election page.

Notice in particular how Sarah Palin's edits really only picked up in the last 6-8 weeks, but User Ferrylodge had edited her page around July 1, before the Aug. 29th nomination.

Unfortunately, because the Toolserver is not very reliable on our queries, we are not always able to serve up live edit data in our dashboard. If you don't get a live dashboard, you can either get at the data from April 2008 that is on our own private database server, or you can wait a while and try again.

* Browse Through Time
Now, you can click on the bars in the dashboard. Clicking on an bar will bring you to the wiki historical context when the edits were made. For Article Dashboard, the system will show all the edits made on the page around the time point you choose. For User Dashboard, WikiDashboard will provide a list of edits that the user made around the time you clicked.

Please let us know if you find any problem or have any feedback. Thanks!

Wednesday, October 15, 2008

User Needs during Social Search

There has been a lot of buzz around social search in the online tech community, but I am largely disappointed by the new tools and services I've encountered. It's not that these sites are unusable, but that they each seem to take on a different conception of what social search is and when/how it will be useful. Have these sites actually studied users doing social search tasks?

Social search may never have one clear, precise definition---and that's fine. However, my instinct is to look at the users and their behaviors, goals, and needs before designing technology. Actually useful social search facilities may be some ways off still (despite the numerous social search sites that advertise themselves as the future of search). First, we need to address some questions, such as:

  1. Where are social interactions useful in the search process?

  2. Why are social interactions useful when they occur?

Study Methods
To answer these questions, Ed Chi & I ran a survey on Mechanical Turk asking 150 users to recount their most recent search experience (also briefly described here and here). We didn't provide grand incentives for completing our survey (merely 20-35 cents), but we structured the survey in a narrative format and figured that most people completed it because it was fun or interesting. (This is a major reason for Turker participation.)

For example, instead of asking a single open-ended question about the search process, we first asked people when the episode occurred, what type of information they were seeking, why they needed it, and what they were doing immediately before they began their search. After this, we probed for details of the search act itself along with actions users took after the search. Our 27-question survey was structured in a before-during-after type format, primarily to establish a narrative and to collect as much detailed information about the context and purpose of users' actions.

We collected responses from 150 anonymous, English-speaking users with diverse backgrounds and occupations. In fact, there was so much diversity in our sample that the most highly represented professions were in Education (9%) and Financial Services (9%). The next ranking professions were Healthcare (7%) and Government Agency (6%) positions. We were quite surprised by the range of companies people worked for: from 1-person companies run out of people's homes to LexisNexis, Liberty Mutual, EA Games, and the IRS!

Our data analysis resulted in a model of social search that incorporated our findings of the role of social interactions during search with related work in search, information seeking and foraging. Without presenting the whole model here, I will highlight the summary points and conclusions from our work. (The full paper is available here.)

Search Motivations
There were two classes of "users" in our sample who we named according to their inherent search motivations. The majority of searchers were self-motivated (69%), meaning that their searches were self-initiated, done for their own personal benefit, or because they had a personal interest in finding the answer to a question. The remaining 31% of users were "externally-motivated"---or were performing searches because of a specific request by a boss, customer, or client.

Not surprisingly, a majority (70%) of externally-motivated searchers interacted with others before they executed a search. The fact that these searches were prompted by other people often led to conversations between the searcher and requester so that the searcher could gather enough information to establish the guidelines for the task. This class of behavior is noteworthy because even though these users engaged in social interactions, they were often required to or may not have otherwise had the occasion to interact.

Although only 30% of self-motivated searchers interacted with others before they executed a search, their reasons for interacting were more varied. While some still needed to establish search guidelines, others were seeking advice, brainstorming ideas, or collecting search tips (e.g., keywords, URLs, etc.). In many cases, these social interactions were natural extensions of their natural search process---these users were performing self-initiated searches afterall. Again this is noteworthy, suggesting that self-motivated searchers would be best supported by social search facilities.

Search Acts
Next, we identified three types of search acts: navigational, transactional, and informational. These classifications were based on Broder’s (2002) taxonomy of information needs in web search, and I'm only going to review our users' informational search patterns (searching for information assumed to be present, but otherwise unknown) since it proved to be the most interesting. Informational search is typically an exploratory process, combining foraging and sensemaking. As an example:
An environmental engineer began searching online for a digital schematic of a storm-water pump while simultaneously browsing through printed materials to get "a better idea of what the tool is called." This search was iteratively refined as the engineer encountered new information, first on and then on Google, that allowed him to update his representation of the search space, or what might be called a "search schema." He finally discovered a keyword combination that provided the desired results.

Over half of search experiences in our sample were informational in nature (59.3%), and their associated search behaviors (foraging and sensemaking) led to interactions with others nearly half the time. Furthermore, 61.1% of information searchers were self-motivated. It appears there is a demand and a desire for social inputs where the search query is undeveloped or poorly specified, and personally relevant.

Post-Search Sharing
Finally, we noticed that, again, nearly half our users (47.3%) shared information with others following their search. This is not wholly unexpected, but points to the need for better online organizational and sharing tools, especially ones that could be built into the web browser or search engine itself. Instead, an interesting finding is why people chose to share information.

Externally-motivated searchers almost always shared information out of obligation---to provide information back to the boss or client who requested the search in the first place. Self-motivated searchers, however, often shared information to get feedback, to make sure the information was accurate and valid, or because they thought others would find it interesting.

Summary and Conclusion
In summary, we classified two types of users in our study: externally-prompted searchers and self-motivated searchers. The self-motivated were the most interesting because of their search habits, propensity to seek help from others, and the reasons behind their social exchanges. For this class of users, a majority performed informational, exploratory searches where the search query was ambiguous, unclear, or poorly specified, leading to a need for guidance from others. Their social interactions, therefore, were primarily used to brainstorm, get more information, and further develop their search schema before embarking on their search. Finally, the search process didn't end after these users identified preliminary search results---they often shared their findings out of interest to others, but also to get feedback, validate their results, and contemplate refining and repeating their search.

It is noteworthy that we did not ask users to report social search experiences in the survey. Instead, we asked for their most recent search act, regardless of what it was, expecting that across all 150 examples we would be able to begin finding generalizable patterns. Indeed, a large majority performed social search acts, but nearly all of the social exchanges were done through real-world interactions---not through online tools. It is no surprise that online tools need to better support social search experiences (our study is only further proof of this); but our study does contribute to a better understanding of user needs during "social" search, which may lead to tools that can best identify and support the class of users and search types best suited for explicit and implicit social support during search.

Finally, in response to the questions I posed at the very beginning:

Where are social interactions useful in the search process?
Before, during, and after a "search act"! Over 2/3 of our sample interacted with others at some point during the course of searching. However, social interactions may not benefit everyone equally---they appear to provide the best support for self-motivated users and users performing informational searches.

Why are social interactions useful when they occur?
It depends! The reasons for engaging with others ranged from a need to establish search guidelines to a need for brainstorming, collecting search tips, seeking advice, getting feedback, and validating search results. Social support during search may be best appreciated and adopted if it directly addresses these types of user needs.

Brynn M. Evans, Ed H. Chi. Towards a Model of Understanding Social Search. In Proc. of Computer-Supported Cooperative Work (CSCW), (to appear). ACM Press, 2008. San Diego, CA.

Thursday, October 2, 2008

CSCW2008 Paper on "Towards a Model of Understanding Social Search"

Search engine researchers typically depict search as the solitary activity of an individual searcher. They hardly ever talk about the social interactions that occurs around search. I think this is just plain wrong.

Brynn Evans and I recently conducted research asking web users their experiences of using search engines on the web. We conducted a type of survey called Critical Incident Survey, where we asked them to recall the last time they did a search on the web, and what that experience was like. Results from our critical-incident survey of 150 users on Amazon’s Mechanical Turk suggest that social interactions play an important role throughout the search process.

We surveyed users about their most recent searching experience. We used Amazon’s Mechanical Turk, a type of micro-task market, which can engage a large number of users to perform evaluation tasks both at low cost and relatively quickly (see our previous published paper in CHI2008 about this approach of doing user studies).

We recruited users with a specific statement of our purpose: "We are interested in how you search for digital information on your computer. Please answer the following questions about your most recent search experience."

We then analyzed the results from the survey and looked to see where social interactions occurred. Note that we didn't specifically ask them to recall incidents in which they had social interactions---just the "most recent" search they did. This style of survey forces users to recall the last significant event that they essentially can still remember. Consequently, about 2/3 of search acts occurred on the same day that users filled out our survey (48.7% occurred “recently” and 14.7% occurred “earlier in the day”). 19.3% of searches occurred the day before, and 17.3% occurred more than 2 days ago.

Here is an example of an interesting report we received. A barista (let's call her Janet) works in a cafe, and couldn't remember a really good recipe for a special drink. But she can remember just several ingredients in the recipe. She asks her colleagues if they know the drink, and of course she didn't know the name of the drink. She had partial knowledge of what she needs to know, but only had more specific information to find the recipe. She goes to Google and types in the ingredients and finally finds recipe after some browsing and searching. After she finds the recipe, she prints out the information and shares it with her co-workers in the cafe the next day.

Interestingly, Janet's extended search process not only extended over a few days, but she also interacted socially around her search process both before as well as after the search. The problem is that Google only sees her interaction with the search engine for a brief period of time, not knowing the entire social process that occurred behind the scene. Perhaps the search engine only saw keywords like "coffee cinnamon honey", but not how she had obtained some of these ingredients' name from other co-workers nor how she printed out the result to share with someone.

Janet never had a chance to interact with other baristas (who might be online at that moment) to see if they had a better idea about how to look for the recipe. Her new found knowledge was also not shared with other like-minded community interested in coffee drinks. Delicious and other social tagging sites can be used by groups of people to share what they have found, but the knowledge does not travel easily from the person who found it to the person that needs it efficiently. It seems tool support for social search is still relatively poor.

Now, our definition of “social search” is intended to be broad, to include a range of possible social interactions that may facilitate information seeking and sensemaking tasks:
“Social search” is an umbrella term used to describe
search acts that make use of social interactions with
others. These interactions may be explicit or implicit,
co-located or remote, synchronous or asynchronous.

In terms of results from our research, this example insight is just the tip of the iceberg. Stay tuned for more results from this research about to be published in CSCW2008.

Brynn Evans, Ed H. Chi. Towards a Model of Understanding Social Search. In Proc. of Computer-Supported Cooperative Work (CSCW), (to appear). ACM Press, 2008. San Diego, CA.