In the fall of 2011, the journal Nature Structural and Molecular Biology published a paper revealing that after more than a decade of effort, researchers had succeeded in mapping the structure of an enzyme used by retroviruses similar to HIV. The achievement was widely viewed as a breakthrough, but there was something else astonishing about the article. Listed among the international group of scientists who had contributed to the discovery was something called the “Foldit Void Crushers Group.” It was the name for a collective of video gamers.

Foldit, a novel experiment created by a group of scientists and game designers at the University of Washington, had asked the gamers — some still in middle school and few with a background in the sciences, much less microbiology — to determine how proteins would fold in the enzyme. Within hours, thousands of people were competing against (and collaborating with) one another. After three weeks, they’d succeeded where the microbiologists and the computers had failed. “This is the first example I know of game players solving a long-standing scientific problem,” David Baker, a Foldit cocreator, said at the time.

It wasn’t to be the last. Foldit has continued to produce accurate models for other highly complex enzymes. Other research projects have similarly tapped the crowd to perform tasks ranging from the simple collection of data to advanced problem solving. One of Foldit’s cocreators, Adrien Treuille, has gone on to develop a similar game, Eterna, in which gamers create designs for synthetic RNA. Eterna’s tagline neatly sums up the project’s central premise: “Solve Puzzles. Invent Medicine.” The designs created by Eterna’s top citizen-scientists are then synthesized at Stanford.

Efforts like these may possibly revolutionize how we treat disease. But they’re holding out another promise as well: a realization that conventional management practice is often dead wrong about who is best suited for a task. The best way to match talent to task, at least in the world of nanobiotechnology, isn’t to assign the fanciest degrees to the toughest jobs, but rather to observe the behavior of thousands of people and identify those who show the greatest aptitude for the cognitive skills that the task requires. “You’d think a PhD in biochemistry would be very good at designing protein molecules,” says Zoran Popović, the University of Washington game designer behind Foldit. Not so. “Biochemists are good at other things. But Foldit requires a narrow, deeper expertise.”

Some gamers have a preternatural ability to recognize patterns, an innate form of spatial reasoning most of us lack. Others — often “grandmothers without a high school education,” says Popović — exercise a particular social skill. “They’re good at getting people unstuck. They get them to approach the problem differently.” What big pharmaceutical company would have anticipated the need to hire under-educated grandmothers? (We know a few seniors in need of work, if Eli Lilly HR is thinking of rejiggering its recruitment strategy.)
Treuille noted that he and his colleagues at Eterna could “filter through hundreds of thousands of people who are experts at very esoteric tasks.” They were able, in other words, to match talent to task with exceptional efficiency — not based on someone’s CV and not based on the magic of self-selection, but rather through the thousands of data points generated by the game. Eterna represents a radical rethinking of one of capitalism’s central assumptions, that labor is best allocated through a command-and-control style of management. Eterna instead relies on an attribute — diversity — that has traditionally been underestimated. Indeed, before the Internet, it often seemed difficult to achieve.

When it works — and contrary to the initial hype, it’s hardly a digital age panacea — crowdsourcing exhibits an almost magical efficacy. Institutions and companies like NASA, the LEGO Group and Samsung have integrated public contributions into the core of how they do business. In the process they’ve remade the boundary that has traditionally separated the producers of a thing from the consumers of that thing. It is now a permeable layer, in which ideas and creativity and even control over such crucial aspects as long-term strategy are a collaborative effort.

The theoretical underpinnings for this approach lie in the nascent discipline of complex systems; the potency of the pixie dust in crowdsourcing is largely a function of the diversity that naturally occurs in any large group of people. And it turns out, science has a history of using distributed knowledge networks to tap diverse talents across disciplines. One of the best-known examples is the Longitude Prize. In 1714 the Parliament of England offered a £10,000 prize to anyone who could figure out a method to determine longitude. Leading scientific minds focused their considerable talents on this problem, but the purse was claimed by the self-taught clockmaker John Harrison.

Of course, amateurs have always made contributions to disciplines like astronomy and meteorology that thrive on large numbers of observations. But before the Internet, the public had little opportunity to contribute to the creation of other types of scientific knowledge. In recent years, a wide range of companies, individuals and academic fields have used our global communications network to increase the sheer brainpower brought to bear on any individual problem and, more important, facilitate a cognitive diversity that is often lacking in the rarefied climes of the corporate or academic laboratory.

Founded in 2000 by the pharmaceutical company Eli Lilly, InnoCentive built its business model on being able to deliver that kind of highly diversified intellectual muscle to its clients. InnoCentive takes difficult problems from major companies, commercial R&D labs and medical research initiatives and posts them to an online bulletin board. These are frequented by nearly 400,000 professional and amateur scientists from some 200 countries, with more than half living outside of the Americas. These are not your run-of-the-mill scientific pickles. If the thousands of chemists at a multinational pharmaceutical company like Merck can’t solve a chemistry problem, you wouldn’t think it’s going to fall to some first-year electrical engineering student at the University of Texas.

And yet … anyone can post a solution. If it works, the individual receives a reward, generally between $10,000 and $40,000. InnoCentive says about 85 percent of the problems are eventually solved, which is quite a batting average given the scale of the challenges. But what’s interesting is who solves the problems, and how.

What’s more remarkable than the fact that more than 60 percent of InnoCentive’s “solvers,” as they’re known, boast a master’s degree or PhD is the fact that nearly 40 percent don’t. In fact, one of the most prolific solvers was a Canadian handyman who had dropped out of his PhD program in particle physics in order to care for his parents.

This isn’t as astonishing as it sounds. Recall that InnoCentive’s challenges are generally filled with problems that smart minds have already failed to solve. If a large consumer products company faces a challenge in economically producing a chemical compound, it’s likely to assign its best chemists to the challenge. We’re inclined to believe that the smartest, best-trained people in a given discipline — the experts — are the best qualified to solve a problem in their specialty. And indeed, they often are. When they fail, as they will from time to time, our unquestioning faith in the principle of “ability” leads us to imagine that we need to find a better solver: other experts with similarly high levels of training.

But it is in the nature of high ability to reproduce itself — the new team of experts, it turns out, trained at the same amazing schools, institutes and companies as the previous experts. Similarly brilliant, our two sets of experts can be relied on to apply the same methods to the problem, and share as well the same biases, blind spots and unconscious tendencies. “Ability matters,” says Scott E. Page, social scientist and author of The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. “But in the aggregate it offers diminishing returns.”

According to research out of Harvard Business School, there’s a positive correlation between successful solutions and what the researcher Karim Lakhani calls “distance from field.” In plain language, the less exposed a given solver is to the discipline in which the problem resides, the more likely he or she is to solve it.

This has actionable implications for how we allocate intellectual capital, or as we may increasingly find, how we allow it to allocate itself, à la projects like InnoCentive or Eterna. Due to an increasing body of research demonstrating that diverse groups are more productive in a wide range of applications, diversity is becoming a strategic imperative for schools, firms and other types of institutions. It may be good politics and good PR and, depending on an individual’s commitment to racial and gender equity, good for the soul. But in an era in which your challenges are likely to feature maximum complexity as well, it’s simply good management, which marks a striking departure from an age when diversity was presumed to come at the expense of ability.

Race, gender, socioeconomic background and disciplinary training are all important, but only inasmuch as they are ciphers for the kinds of life experiences that produce cognitive diversity. Further, because we can’t know in advance which of those diverse backgrounds, educational experiences or intellectual tendencies might produce a breakthrough, said Page, “we should think of our differences as forms of talent. To leverage that talent requires patience and practice.”

Excerpted from the new book Whiplash: How to Survive Our Faster Future by Joi Ito and Jeff Howe. Copyright © 2016 by Joichi Ito and Jeff Howe. Reprinted with permission of Grand Central Publishing. All rights reserved.