Raluca Budiu, who is a post-doc working in our group, has conducted some very interesting research with us on how tagging appears to affect human information processing. She studied two techniques for producing tags: (1) the traditional type-to-tag interface of typing keywords into a free-form textbox after reading a passage or article; (2) a PARC-developed click2tag interface that allows users to click on keywords in the paragraph to tag the content.
The experiment consisted of 20 subjects and 24 passages in a within-subject design. Participants had to first study passages and tag them, and then they performed memory tests on what they had actually read and tagged. The memory tasks were that, after tagging the content, they have to either (a) freely recall and type as many facts from the passages as possible; or (b) answer 6 true/false sentences in a recognition task.
As reported in the paper, the results suggest that:
- In the type-to-tag condition, users appears to elaborate what they have just read, and re-encoded the knowledge with keywords that might be helpful for later use. This appears to help the free-recall task (a) above. In other words, users seem to end up with a top-down process and induces them to schematize what they have learned.
- While in the click2tag condition, users appears to re-read the passages to pick out keywords from the sentences, and this appears to help them in their recognition tasks (b) above. In other words, users seem to use a bottom-up process that simply picked out the most important keywords from the passage.
Click here to download the technical report and pre-print (the highlights in the paper are mine).
7 comments:
From quickly looking at your results, it appears that there is no real mnemonic advantage to tagging relative to no tagging. Interesting - as someone who has studied memory, I would have predicted that typing would help recall and that clicking would have helped recognition, but I would have thought there would have been a general advantage in recall to the reprocessing of information by tagging. Were these people experienced users of tagging apps?
Chris,
You're referring to the encoding hypothesis in reading research, and I'm not sure if our experiment fully illuminates what's going on there. Remember that this is testing right after they've tagged, not several weeks later, so this is no time-delay. We're doing the follow-on study where there is a time-delay.
It's also not clear what role (if any) does expertise play in this circumstance.
Chris, my expectation entering this study was also that tagging in general would help recall. The data in fact do suggest exactly this -- in Blocks 1 and 2 we get that the tagging conditions combined are significantly better than the no tagging condition. We are also currently collecting more data, which should increase the statistical power of the analysis and should lead to even stronger results.
To respond to your specific question, the participants were not necessarily expert taggers, and that showed up at the begining of the experiment (Block 0), where they took some time to do the task and were actually better in the no-tag condition at recall.
Thanks for the thoughtful replies. I think this is a very interesting line of research. I thought that expertise might play a role in this task because I find that I intentionally use tagging in del.icio.us as a quick and easy 'memory extension' and probably attend less to those things because I know they will be easy to find later via the tags I supply. If you don't mind, I may have another question or two after reading this work a little more closely.
Oh, and so I'm not totally anonymous here, Ed - I'm Chris Hadley from wikiHow.
Thanks, Chris, for your comments. We love WikiHow, so let's definitely pay attention to each other's work. BTW, you should come to some of our Web2.0 speaker events. It would be good to see you there tomorrow if you can come. Ross Mayfield is speaking.
thanks very much for sharing these results. i'd like to know if the conclusions are the same when the thing being tagged is an image not a text. within the steve.museum collaboraration [some papers online] we're looking at whether tagging could improve access to online art collections. it would be interesting to repeat the experiment with works of art.
Raluca, I am merely thinking out aloud here, so please accept my contribution here in that spirit.
Isn't tagging the online equivalent of the magazine pile?
My interest in this subject is solely at the individual level. Of course I am interested in any kind of augmentation that assists improved memory.
How do you separate social tagging from intelligent, or innovative use, from what ordinarily is simply a collecting activity? Please answer this, I utilize rhetorical questions as a part of my own process.
I come from this from a pure raw human viewpoint because when I read articles I tend to think out aloud, so this is not a comment but purely an observation which is my individual way of thinking about what I have just read or happen to google in the process of read this.
Without an intelligent end user approach to tagging, without the user thinking about his own tagging and without teaching people how to tag (and how many actually want to learn this), then IMHO tagging simply increases information and information overload has IMHO no relationship to effective memory techniques.
Personally I try to eliminate the magazine pile and where possible the tag, for I consider raw involvement, visceral observation and utilizing personal database resource as my personal information strategy.
Visiting here and doing the usual follow up led me to learn about Douglas Englebart and at some future point I will create time to learn more about him, so my brief glimpse of the world you reside in has been interesting, but as a layman I look at social tagging from a purely personal point of view - how it improves my own organization and thinking - and I find the relationship between direct thinking and memory to be of higher importance than the relationship between tagging and memory, which is obviously what I am in the process of thinking out aloud here. I want to dig the gems out of my own thinking later rather than simply tag thoughts.
From a street-smart view, I do think that tagging is a skill that the user can learn and as a user,
I am interested in lifestyle improvement rather than social theory - the core rhetorical question I leave myself as I contemplate here is how to bridge the gap between my tagging use as a social habit and as an individual skill?
M.
Post a Comment