Jonah Lehrer had a nice column in the weekend Wall Street Journal about the "wisdom of crowds" - the notion that we can get the right answers to tough problems by marshaling the collective intellect from a large group of people through a mechanism such as a voting process. Lehrer points to a study by Swiss scholars that suggests that crowds may not always be so smart. Lehrer writes:
The experiment was straightforward. The researchers gathered 144 Swiss college students, sat them in isolated cubicles, and then asked them to answer various questions, such as the number of new immigrants living in Zurich. In many instances, the crowd proved correct. When asked about those immigrants, for instance, the median guess of the students was 10,000. The answer was 10,067.
The scientists then gave their subjects access to the guesses of the other members of the group. As a result, they were able to adjust their subsequent estimates based on the feedback of the crowd. The results were depressing. All of a sudden, the range of guesses dramatically narrowed; people were mindlessly imitating each other. Instead of canceling out their errors, they ended up magnifying their biases, which is why each round led to worse guesses. Although these subjects were far more confident that they were right—it's reassuring to know what other people think—this confidence was misplaced.
The results do not surprise at all. The key to the wisdom of crowds, as James Suroweicki explained in his great book on the phenomenon, is that individuals must make independent judgments. In a voting process, where individuals do not interact prior to rendering their judgment, we have independence. However, in group and organization settings, social influence often plays a powerful role. People are affected and even swayed by the judgments of others, and that often has a negative effect on the "wisdom" of the crowd. I've always highlighted this fact by asking people whether they think the audience's accuracy on the game show Who Wants to be a Millionaire would go up or down if people had time to sit in a conference room and deliberate about the particular trivia question. Because of social influence, I have always argued that accuracy would likely fall. This simple experiment reinforces that conclusion.
The experiment was straightforward. The researchers gathered 144 Swiss college students, sat them in isolated cubicles, and then asked them to answer various questions, such as the number of new immigrants living in Zurich. In many instances, the crowd proved correct. When asked about those immigrants, for instance, the median guess of the students was 10,000. The answer was 10,067.
The scientists then gave their subjects access to the guesses of the other members of the group. As a result, they were able to adjust their subsequent estimates based on the feedback of the crowd. The results were depressing. All of a sudden, the range of guesses dramatically narrowed; people were mindlessly imitating each other. Instead of canceling out their errors, they ended up magnifying their biases, which is why each round led to worse guesses. Although these subjects were far more confident that they were right—it's reassuring to know what other people think—this confidence was misplaced.
The results do not surprise at all. The key to the wisdom of crowds, as James Suroweicki explained in his great book on the phenomenon, is that individuals must make independent judgments. In a voting process, where individuals do not interact prior to rendering their judgment, we have independence. However, in group and organization settings, social influence often plays a powerful role. People are affected and even swayed by the judgments of others, and that often has a negative effect on the "wisdom" of the crowd. I've always highlighted this fact by asking people whether they think the audience's accuracy on the game show Who Wants to be a Millionaire would go up or down if people had time to sit in a conference room and deliberate about the particular trivia question. Because of social influence, I have always argued that accuracy would likely fall. This simple experiment reinforces that conclusion.
