The age-old maxim, Vox Populi, Vox Dei meaning Voice of People is the Voice of God may hold some credence against the popular belief that crowd behavior is the expression of human madness. Lets see how.

Wisdom of the crowd phenomenon was first noted just over a century ago by Sir Francis Galton when he asked 787 people to guess the weight of an ox at a country fair. None of them got the right answer, but, pooled together, their collective guess was almost perfect – The ox actually weighed 1,198 lbs. and the collective average guess was 1,197 lbs.

Wisdom of the crowd was also seen on the game show ‘Who Wants to be a Millionaire’. The ‘Audience Poll’ option was correct more than 90% of the time.

Best Buy an electronic retail store implemented sales forecasting system called TagTrade and got all its employees to forecast holiday sales. The accuracy rate was 99.5%. Significantly better forecast than what the company obtained from experts by paying a lot of money.

This phenomenon got a lot more attention in the last decade when James Surowiecki’s book ‘The Wisdom of Crowds” became a best seller. I first learnt about this concept when I took the online course Model Thinking by Scott E Page and my interest in the topic attracted me to James Surowiecki’s book.

James Surowiecki writes –

“Under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them”

Four conditions should exist for the ‘Wisdom of the crowd’ phenomenon to work –

• Diversity – The group should consist of people with varying degree of knowledge / insight. Each person should have some private information, even if it’s just an eccentric interpretation of the known facts. Such diversity in bringing together bits of information contributes to the rich whole. Groups of experts don’t do as well because their education and experience tend to be common and make them act and think alike.
• Independence – People’s opinions should not be determined by the opinions of those around them. Too much of the same information leads to herding behavior and herding creates stock market bubbles, market crashes and other manias.
• Decentralization – People should be able to opine based on local knowledge. Decentralization helps ensure independence and diversity that groups need to be wise. In the context of politics / democracy, if people voted only based on the national capital issues and not based on local issues, then the risk of selecting a wrong candidate is extremely high.
• Aggregation – Some mechanism should exist for turning decentralized / dispersed opinions into a collective decision. Without some sort of aggregator (e.g. marketplace, election / voting systems, survey technology, etc.), all the above three conditions are useless and the group intelligence is lost.

In essence it’s just a mathematical truism. Each person’s guess, has two components a) information and b) error. A large unbiased sample size cancels out errors on the extreme ends and you’re left with the information. That’s why the wisdom of the crowd is in the collective average.

What are some of the notable practical applications of this phenomenon?

Search engines – Bing / Google aggregates and disseminates information based on this model. Relevant links to websites are not formed deliberately by experts, but established upon successful search query completion by millions of users on an almost real-time basis.

Democracy – Aristotle argues “If liberty and equality, as is thought by some, are chiefly to be found in democracy, they will be best attained when all persons alike share in government to the utmost”. Democracy works because of wisdom of the crowd.

Online marketplaces and knowledge portals – Amazon.com, ebay.com and many other websites consolidate feedback / opinion of the users of its products and services. Availability of such intelligent user feedback in-turn attracts more customers and that helps generate more sales. Wisdom of the crowd is what drives sales on these online marketplaces. Wikipedia is the embodiment of wisdom of the crowd.

Jury system of judgement – It’s the jury and not the judge that determines the fate of court cases in US and many other democratic countries. Jury system is a classic example of relying on wisdom of the crowd for fair judgment.

Good Judgment Project – Philip Tetlock a pioneer in the field of forecasting, decision-making and judgement is tapping into wisdom of the crowd phenomenon and over the past few years has outperformed intelligence officers having access to classified information as it relates to forecasting geo-political events. You can participate in this project too.

Ideas generation and forecasting in large companies – Bill Gates used to conduct ‘Think Week’ while at Microsoft to solicit ideas from all his employees. Many of the research projects were influenced by ‘Think Week’ ideas. The recent upsurge in crowdsourced funding is another off-shoot of this phenomenon.

When will this not work?

Too many smart people – Dumbness is ok. They tend to cancel out errors. Too many smart people tend to think alike and they skew the benefits of wisdom of the crowd.

Specialized task requiring subject matter expertise – If you have a plumbing problem, you will not get better results if you bring together a group of Doctors, Accountants, Military Generals, Management Consultants etc. All you need is a good Plumber to fix your problem. Diversity and independence will not work in such situations.

Strong centralized or top down influence – If people are told what to do and are heavily influenced by an authoritative figure who is either revered or feared, then this phenomenon will not work. Dictatorial regimes, bureaucratic companies / organizations, cult following, etc. are cases where one cannot benefit from wisdom of the crowd because there exists none.

Watch this BBC YouTube video and I am sure you will be amazed with the magic of wisdom of the crowd demonstrated using jelly beans.

Average often signifies mediocrity but when it comes to decision-making, it often signifies excellence. Humans gained evolutionary advantage not working as individuals but cooperating as groups. It looks like we are programmed to excel not as individuals but as a groups consisting of diverse and independent individuals.