I'm in Hong Kong on some personal business and have had some alone time to think about our research direction. One of the things we have been doing lately at PARC is understanding more about the past work on collaboration, and how it might be changed (or not) by Web2.0 design principles. We have been talking to Gary and Judy Olson, who are recognized experts in collaboration systems and models for large science remote laboratories formed by scientists across many institutions. These laboratories (called collaboratories by the Olsons) are a great way to understand what works and what doesn't work in the real world, when CSCW and distant collaboration technologies are put to the test and used in real everyday scientific work. These studies are interesting because they're real 'living laboratories' and scientists engaged in these collaborations because it is necessary to do real work.
Interestingly, one of the best articles that summarizes their work is written by Technology Review (found here). Studying more than 200 collaboratories, the Olsons found that there are a number of pre-requisites for successful collaboration:
- Make sure your research community is ready
- Tackle big questions
- Get each individual participant on board
- Gear up for major technical challenges
- Put enough resources into project management
- Establish a common vocabulary
- Patience, visionary planning and stable management.
What's perhaps most interesting about this list is the amount of common sense it contains, and how it would be impossible to escape these pre-requisites even in Web2.0 collaboration systems. I find it interesting intellectual exercise to apply these requirements to successful Web2.0 systems (such as Wikipedia, delicious, and digg) to see if they meet these requirements.
Sunday, May 17, 2009
Friday, April 24, 2009
Social bookmarks as traces left behind as navigational signposts
Social tagging arose out of the need to organize found content that is worth revisiting. It is natural therefore to think of social tagging and bookmarking as navigational signposts for interesting content. The collective behavior of users who tagged contents seems to offer a good basis for exploratory search interfaces, even for users who are not using social bookmarking sites.
In Boston at the CHI2009 conference, we presented a paper that showed how our tag-based search interface called MrTaggy can be used as learning tools for people to find content relating to a particular topic. We have already announced its availability on this blog, and also touched upon the way in which it is implemented. Here we will briefly blog about an evaluation study we did on this system in order to understand its learning effects.
Short Story:
The tag-based search system allows users to utilize relevance feedback on tags to indicate their interest in various topics, enabling rapid exploration of the topic space. It turns out that the experiment shows that the system seems to provide a kind of scaffold for users to learn new topics.
Long Story:
We recently completed a 30-subject study of MrTaggy [see reference below for full detail]. We compared the full exploratory MrTaggy interface to a baseline version of MrTaggy that only supported traditional query-based search.
We tested participants’ performance in three different topic domains.
The results show:
(1) Subjects using the MrTaggy full exploratory interface took advantage of the additional features provided by relevance feedback, without giving up their usual manual query typing behavior.
(2) For learning outcomes, subjects using the full exploratory system generally wrote summaries of higher quality compared to baseline system users.
(3) To also gauge learning outcomes, we asked subjects to generate keywords and input as many keywords as possible that were relevant to the topic domain in a certain time limit. Subjects using the exploratory system were generally able to generate more reasonable keywords than the baseline system users.
(4) Finally, other convergent measures show that they also spent more time on the learning tasks, and had a higher cognitive load. Taken together with the higher learning measure outcomes, the users appear to be more engaged in exploration than the participants using the baseline system.
Our findings regarding the use of our exploratory tag search system are promising. The empirical results show that subjects can effectively use data generated by social tagging as “navigational advice” in the learning domain.
The experimental results suggest that users’ explorations in unfamiliar topic areas are supported by the domain keyword recommendations presented in the related tags list and the opportunity for relevance feedback.
Since social search engines that depend on social cues rely on data quality and increasing coverage of the explorable web space, we expect that the constantly increasing popularity of social bookmarking services will improve social search browsers like MrTaggy. The results of this project point to the promise of social search to fulfill a need in providing navigational signposts to the best contents.
Reference:
Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 625-634.
ACM Link
Talk Slides
In Boston at the CHI2009 conference, we presented a paper that showed how our tag-based search interface called MrTaggy can be used as learning tools for people to find content relating to a particular topic. We have already announced its availability on this blog, and also touched upon the way in which it is implemented. Here we will briefly blog about an evaluation study we did on this system in order to understand its learning effects.
Short Story:
The tag-based search system allows users to utilize relevance feedback on tags to indicate their interest in various topics, enabling rapid exploration of the topic space. It turns out that the experiment shows that the system seems to provide a kind of scaffold for users to learn new topics.
Long Story:
We recently completed a 30-subject study of MrTaggy [see reference below for full detail]. We compared the full exploratory MrTaggy interface to a baseline version of MrTaggy that only supported traditional query-based search.
We tested participants’ performance in three different topic domains.
The results show:
(1) Subjects using the MrTaggy full exploratory interface took advantage of the additional features provided by relevance feedback, without giving up their usual manual query typing behavior.
(2) For learning outcomes, subjects using the full exploratory system generally wrote summaries of higher quality compared to baseline system users.
(3) To also gauge learning outcomes, we asked subjects to generate keywords and input as many keywords as possible that were relevant to the topic domain in a certain time limit. Subjects using the exploratory system were generally able to generate more reasonable keywords than the baseline system users.
(4) Finally, other convergent measures show that they also spent more time on the learning tasks, and had a higher cognitive load. Taken together with the higher learning measure outcomes, the users appear to be more engaged in exploration than the participants using the baseline system.
Our findings regarding the use of our exploratory tag search system are promising. The empirical results show that subjects can effectively use data generated by social tagging as “navigational advice” in the learning domain.
The experimental results suggest that users’ explorations in unfamiliar topic areas are supported by the domain keyword recommendations presented in the related tags list and the opportunity for relevance feedback.
Since social search engines that depend on social cues rely on data quality and increasing coverage of the explorable web space, we expect that the constantly increasing popularity of social bookmarking services will improve social search browsers like MrTaggy. The results of this project point to the promise of social search to fulfill a need in providing navigational signposts to the best contents.
Reference:
Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 625-634.
ACM Link
Talk Slides
Subscribe to:
Posts (Atom)