Blog Category // Collective Intelligence
When two heads are better than one … The problem of optimally aggregating diverse opinions together to achieve the best result available is hardly new. The Marquis de Condorcet, 1785, and Francis Galton, 1907, were the first to show how crowds could outperform single individuals as well as experts. More recently prediction markets and Web companies like UNU® have inherited this legacy by finding effective ways to aggregate opinions to forecast successfully future events. Most of the success of these systems can be attributed to one simple mechanism. If we assume people are independent in their judgements, the errors of individuals will be uncorrelated from each other. In other words, some people will over-estimate the true answer, while some others will underestimate it. Averaging together multiple opinions will give however a better estimate because uncorrelated noise will average out, leaving the signal (i.e. the true value) emerge.
…and when two cooks can spoil the broth So why haven’t we found a better way to make smart collective choices? How is it possible that we often observe crowds getting really wrong answers? Think for example about market bubbles, racism and conformity. The reason behind these phenomena is that most of the assumption of Condorcet and Galton’s theorems are often unmet in reality. People are often not independent in their opinions, we browse the same websites, get our news from the same outlets, we influence each other in unpredictable ways. A rumour might be amplified exponentially or important information could remain not listened to, without any clear understanding of why this is the case. This is why almost nobody among pundits and scientific experts had predicted the outcome of recent political votes. People’s opinions are very difficult to forecast because they are not only based on facts, but also on others’ opinions – which are interconnected by an intricate network of mutual dependencies. Can our scientific knowledge of opinion aggregation help us in avoiding such large-scale social cascades?
Opinion formation and collective action in small groups Although we don’t yet know the precise mechanisms underlying opinion formation at large-scale, our knowledge is remarkably accurate when it comes to small groups of interacting people. In 2010, Bahrami and colleagues from the University College London showed that when people interact with each other they outperform even the best member of the group. But this effect is found only when people share their confidence in their judgements. Confidence plays a very important role in opinion aggregation as it represents the (inverse) uncertainty of the evidence supporting an opinion. By sharing confidence levels, the group is thus able to select, every time, the opinion supported by the strongest evidence. The following year Lorenz and colleagues found that knowledge about estimates of others (but not their confidence!) narrows the diversity of opinions to such an extent that it undermines the wisdom of crowd. In my PhD research I showed that confidence sharing has interesting unexpected by-products. First, people use their own opinions to understand how reliable the information they receive is when no other source of objective feedback is available. This leads people who share the same biases to agree with each other more often, thus reinforcing their views even further. Second, I show that dynamically interacting with others (think about discussing your political views in real life with a friend) greatly differs from simply receiving the same information but without the possibility to interactively discuss (think about reading the same information on the newspaper). Interaction leads to confidence escalation and can potentially produce big errors done with high confidence. This is interesting if we consider that the very same amount of evidence is available in the two conditions.
people use their own opinions to understand how reliable the information they receive is when no other source of objective feedback is available.
Opinion formation and collective action in large groups Researchers have still to link this micro-scale knowledge to macro-scale interventions. Importantly two issues need to be addressed:
Addressing the first point is relatively easy. In theory, when group size increases, confidence should weigh less and a majority rule like behaviour should be more indicative of the group’s final choice. Although experimental evidence is still needed, the emergence and quick take off of massive online experimentation (e.g. Amazon Mechanical Turk) might soon bridge the existing gap between theory and experimental evidence. Addressing the second issue might result more challenging. Most experiments to date have involved simple decisions. A simple decision can be defined as a decision where people have to find the optimal answer along one decision variable. Estimating how many people are in a bar you are walking in might be difficult but it’s a simple decision because it involves only one variable (the number of people) that is not dependent on other variables, like the weather or what the bartender had for dinner. Most decisions in real life – think about buying a house or making a new policy – however are complex, meaning that choosing the right answer involves taking into consideration a number of factors and circumstantial evidence at the same time. For example, when buying a house we cannot rely on the price of the house to know whether it is worth the purchase. This has to be considered while taking into account also the size, the position, the neighbourhood and so on. Few studies so far have explored the realm of group decisions in complex decision space. One notable example is the work. of Ashish Goel and David T. Lee at Stanford University. These researchers have produced mathematical simulations to find optimal ways to aggregate opinions about complex issues (e.g. making a new policy). They show that a simple majority rule might not be the best strategy available to reach the best outcome. More theoretical and empirical work is however still needed.
Every Brutal Choice Has Elegance