Unwise Crowds?
Many of us who believe in Web 2.0 (the concept, not the buzzword) have come to accept the wisdom of crowds like an article of faith. The Frontal Cortex describes a Columbia University sociology experiment that might undermine our dogma (apparently I missed it in Science):
In our study, published last year in Science, more than 14,000 participants registered at our Web site, Music Lab (www.musiclab.columbia.edu), and were asked to listen to, rate and, if they chose, download songs by bands they had never heard of. Some of the participants saw only the names of the songs and bands, while others also saw how many times the songs had been downloaded by previous participants. This second group — in what we called the “social influence” condition — was further split into eight parallel “worlds” such that participants could see the prior downloads of people only in their own world. We didn’t manipulate any of these rankings — all the artists in all the worlds started out identically, with zero downloads — but because the different worlds were kept separate, they subsequently evolved independently of one another.
This setup let us test the possibility of prediction in two very direct ways. First, if people know what they like regardless of what they think other people like, the most successful songs should draw about the same amount of the total market share in both the independent and social-influence conditions — that is, hits shouldn’t be any bigger just because the people downloading them know what other people downloaded. And second, the very same songs — the “best” ones — should become hits in all social-influence worlds.
What we found, however, was exactly the opposite. In all the social-influence worlds, the most popular songs were much more popular (and the least popular songs were less popular) than in the independent condition. At the same time, however, the particular songs that became hits were different in different worlds, just as cumulative-advantage theory would predict. Introducing social influence into human decision making, in other words, didn’t just make the hits bigger; it also made them more unpredictable.
I don’t accept this one study as proof that the entire user-centric content rating system is a failure (or random), but it does highlight some of the perils of a “winner takes most” mode of cultural evolution.
Steve Laniel Apr 20
This study seems to be a restatement of the “recognition heuristic”, which is covered in some depth in Gigerenzer, Todd et al.’s “Simple Heuristics that Make Us Smart”.
Note also that some variant of the recognition heuristic is probably the reason why I linked to the Wikipedia entry above. I got that link by googling for “recognition heuristic,” then picking the top link without really looking at any of the others. The top link happened to be the Wikipedia, which made it more likely that I’d click. But in a certain sense, doing so was irrational: surely I don’t believe that *just* because it’s a Wikipedia entry, it’s the best available web page on the topic. So why did I link to it? I can see arguments both for and against doing so.
This is something we all do, seems to me. It’s why items at the top of Google searches probably stay there.
So I submit that a) the study doesn’t say much that’s new, and b) just gives a bit more support to something we already knew we do.
adam Apr 20
I think what the study shows is that there may be some randomness in what comes out on top. It doesn’t surprise me that people tend to go with what’s already on top — what surprised me, at least somewhat, is that different results come out on top in different universes without any apparent reason.
Karl Sokol Apr 28
An important factor that needs to be considered in this discussion is that the subject of the study was music. Most would agree that the God-given function (or primitive function if you want to leave God out of it) of music has something to do with community. Whatever the communal function is is hotly debated, but the fact that music develops in community isn’t.
So, since the function of music is tied to community, then it is not surprising that the perception that others within a given community have about what others in the community like influences the popularity of a given band. This is different than the belief that a community can do a better job than an individual at discerning something that exists outside of the community (e.g. the weight of a cow or the most efficient word processor.)
It would be interesting to see this exact same study done with software downloads or eye-candy.