Fascinating article earlier this year in “Current Directions in Psychological Science”, examining how the structure of communication in a group can affect the rate at which innovations occur. To quote the EurekAlert! story:
Good ideas can have drawbacks. When information is freely shared, good ideas can stunt innovation by distracting others from pursuing even better ideas, according to Indiana University cognitive scientist Robert Goldstone.
“How do you structure your community so you get the best solution out of the group?” Goldstone said. “It turns out not to be effective if different inventors and labs see exactly what everyone else is doing because of the human tendency to glom onto the current ‘best’ solution.”
[…]
This study used a virtual environment in which study participants worked in specifically designed groups to solve a problem. Participants guessed numbers between 1 and 100, with each number having a hidden value. The goal was for individuals to accumulate the highest score through several rounds of guessing. Across different conditions, the relationship between guesses and scores could either be simple or complex. The participants saw the results of their own guesses and some or all of the guesses of the others in their group.
In the “fully connected” group, everyone’s work was completely accessible to everyone else — much like a tight-knit family or small town. In the “locally connected” group, participants primarily were aware of what their neighbors, or the people on either side, were doing. In the “small world” group, participants also were primarily aware of what their neighbors were doing, but they also had a few distant connections that let them send or retrieve good ideas from outside of their neighborhood.
Goldstone found that the fully connected groups performed the best when solving simple problems. Small world groups, however, performed better on more difficult problems. For these problems, the truism “The more information, the better” is not valid.
“The small world network preserves diversity,” Goldstone said. “One clique could be coming up with one answer, another clique could be coming up with another. As a result, the group as a whole is searching the problem space more effectively. For hard problems, connecting people by small world networks offers a good compromise between having members explore a variety of innovations, while still quickly disseminating promising innovations throughout the group.
The original article is behind a publisher paywall, but here’s a link for those with access.
Many thoughts immediately arise:
- Mark Newman has shown that the structure of many scientific collaboration networks is a small world network, at least as of 2000.
- The “small world” classification is pretty coarse. It’d be helpful to say more precisely which network structures give rise to or inhibit innovation.
- The problems posed by Goldstone and collaborators were pretty artificial. What happens for more realistic problems?
- How does problem-solving effectiveness scale with the size of the group? I expect that the point of diminishing returns is hit pretty quickly even with very difficult problems. A good understanding of this would potentially have implications for science funding.
- Certain periods of intellectual history were especially fertile. What’s the pattern of collaborations look like for the founders of quantum mechanics (or insert your favourite topic)? Is it special?
- More generally, what predictive power does the pattern of collaborations (or citations, or any other type of linkage) have?
This is very interesting. I’ll put in another possible factor to take into account is the costs associated with communication. Another reason for the distraction in a fully connected network could be the attention spent on the communication itself.
Still, we are very far away from the Borg hive mind :), there is no possible way for anyone to keep up with all that is being developed in any given field so there is still a lot of room for originality.
Interesting, thanks for the references!
This is another great example of a common theme in studies of the products of collaborative work. James Surowiecki’s book ‘The Wisdom of Crowds’ touches on a number of very similar examples of this same pattern, highlighting the importance of avoid excessive communication during the early stages of group work.
Its interesting that the phenomenon is clearly not limited to human interactions, you see it in more mathematical situations as well. For example, when constructing classifiers, some of the most effective techniques from machine learning succeed by finding ways to take advantage of a diversity of different sub-models – the more diverse the better. See e.g. bagging – http://en.wikipedia.org/wiki/Bootstrap_aggregating .
Without an opportunity for a diversity of ideas (or models or organisms) to develop, stagnation in local maxima seems to be the consequence.
Interesting.
Btw. Your link is pointing to access through uwaterloo.ca. The generic link is:
http://www.blackwell-synergy.com/doi/abs/10.1111/j.1467-8721.2008.00539.x
Thanks Jonathan – fixed!
Thanks for the comments and the link, Ben, very interesting indeed. If you enjoyed Surowiecki’s book, you might also like Cass Sunstein’s “Infotopia”, which touches on similar topics.
Thanks, I will certainly have a look at it. The same idea that Pedro mentioned (“… there is no possible way for anyone to keep up… “) has been floating around in my head as well. At one level, I wonder if the level of global communication is now or might someday reach a point where overall creativity decreases along the lines of the thread here. If everyone could know everything that everyone else was thinking, we could end up getting stuck (sort of like we very nearly are already when it comes to the spread of diseases). Is it only our limited ability, as humans, to find and assimilate knowledge that will prevent this?
In my experience the biggest problem with sharing scientific information is the risk that someone will attack an idea out of spite or just reflexive competitiveness.
What is interesting about about the current self-selected group involved in Open Science is that the vast majority have proven to be very supportive.
The cynical ones are too busy hoarding their data to explore the blogosphere 🙂
We might be too limited as individuals to grasp any significant portion of what we already know. We have increased the pace of research by specialization and there is typically many more people working within a field than those serving as bridges.
One thing that is improving tremendously is our capacity to find and dig into a part of knowledge that is new to us as individuals. This could in this context as problematic as knowing. If I am faced with a problem to solve I will typically search for the current solutions or best directions for solving this problem. If there is a better solution at the end of a totally different direction then I might not try it.I wonder if this could be tested by machine learning or some type of model (different agents, search accuracy ,etc).