Connecting scientists to scientists
I’ve been struggling for some time with a writing problem. This is the problem of finding a really sharp way of conveying one of the most powerful ideas of open science: all the untapped creative potential existing in latent connections between scientists, and which could be released using suitable tools to activate the most valuable of those latent connections. I’ve discussed this idea in previous essays, but something was always lacking. In this post I take another shot at it, this time confronting the problem head on.
A fact of any scientist’s life is that you carry a lot of unsolved problems around in your head. Some of those problems are big (“find a quantum theory of gravity”), some of them are small (“where’d that damned minus sign disappear in my calculation?”), but all are grist for future progress. Mostly, it’s up to you to solve those problems yourself. If you’re lucky, you might also have a few supportive colleagues who can sometimes help you out.
Very occasionally, though, you’ll solve a problem in a completely different way. You’ll be chatting with a new acquaintance, when one of your problems (or something related) comes up. You’re chatting away when all of a sudden, BANG, you realize that this is just the right person to be talking to. Maybe they can just outright solve your problem. Or maybe they give you some crucial insight that provides the momentum needed to vanquish the problem.
Every working scientist recognizes this type of fortuitous serendipitous interaction. The problem is that they occur too rarely.
A few years ago, I started participating in various open source forums. Over time, I noticed something surprising going on in the healthiest of those forums. When people had a problem that was bugging them, rather than keeping silent about it, they’d post a description of the problem to the forum. Often, I’d look at their question and think to myself “yeah, I can see why they posted, that looks like a tough problem.” Then, forty minutes later, someone would come in and say “Oh, that’s easy, you just do X, Y, and Z”. Very often, X, Y and Z were quite ingenious, or at the least relied on knowledge that neither I nor the original questioner possessed. The original problem had been trivial all along.
What’s going on is similar to the fortuitous scientific exchange. A problem that’s difficult or impossible for most people can be trivial or routine to just the right person. But what was interesting and surprising about the open source forums was this: it seemed to be happening all the time. People who I’d never heard of would pop up, ask an interesting question, then someone else I’d never heard of would pop up, and provide an insightful answer. It didn’t happen every time, but it was happening over and over again.
A big “ahah!” moment for me occurred when I understood what was going on. By scaling up the creative conversation, those open source projects were providing a systematic mechanism that enabled people to find other people with just the right expertise to make their problem easy. Most of us spend much of our time stymied by problems that would be routine, if only we could find the right person to help us. As recently as 20 years ago, finding that right person was likely to be difficult. But what open source forums show is that it is possible to scale up conversation in this way, and significantly increase the likelihood of such serendipitous interaction.
Needless to say, scientists mostly don’t work this way. Many skeptics of open science say they never could, that scientists will forever be unwilling to share their problems and ideas in the way necessary to make this work. For the present post, it’s fine if you hold that position, for my purpose here isn’t to discuss the practicality of doing this. That’s a post for another day.
The question I’m concerned with is, instead, what is lost because we don’t do this? How much do we lose because so many scientists waste their time struggling with problems that some other scientist would find entirely routine?
I don’t know how to answer these questions quantitatively. What I do know is that as a practicing scientist, much of my time was spent working on problems that were hard for me, yet which I absolutely knew would be routine for someone else. The time I spent working on such problems was time lost to the whole scientific enterprise. Yet the tools and culture of science were such that I couldn’t easily outsource those problems to a person with a comparative advantage over me. When I talk about topics like restructuring expert attention, collaboration markets and open source research, this is what I’m talking about: tools and norms which allow us to trade in expert attention, and so to concentrate in areas where we have a comparative advantage.
Now, there are many caveats to this story. Most open source projects fail. Many problems – including many of the “big problems” of science – are intrinsically non-routine, and it may be extremely difficult to identify who (if anyone) has a comparative advantage in solving such problems. Furthermore, even for routine problems, there may be considerable intrinsic transaction costs associated with trade in expert attention – finding a common language, coming to a common understanding of the problem, and so on. The market for a problem may be thin (“find the screwdriver yourself!”) – for example, many of the problems facing benchtop experimentalists are problems exclusive to their own laboratories. Finally, finding ways to successfully scale up scientific conversation is not at all trivial. These are all important caveats, deserving extended discussion in their own right. Despite this, I believe the key idea – developing tools to aggregate information about comparative advantage, and to connect people who might benefit from a trade in attention – is worth taking seriously.
I started this post off with a discussion of the difficulty of describing what I believe is a latent potential for discovery within the scientific community. As I finish the post off, I must say that the post falls short of the strength and sharpness I’d like. What’s really needed is a detailed example that shows the mechanics of open source in action: how the dynamic division of labour actually works in a successful open source project. At present, so far as I’m aware there are no really successful examples within science; the culture of science remains too closed. There are, however, some extremely encouraging nascent examples, like open notebook science, and open source biology, and one day hopefully these and others will bloom.
Eric Raymond’s classic essay The Cathedral and the Bazaar and Yochai Benkler’s article Coase’s Penguin both contain illuminating discussions of open source, developing many of the ideas discussed in this post much further. The best way to understand open source, though, is to get actively involved in one of the many healthy projects. Cameron Neylon has a cautionary post (see also here) about the difficulty of building effective networking tools for scientists. Many people have written about ways of better connecting scientists – I especially enjoyed Shirley Wu and Cameron Neylon’s essays on the subject.
I’m writing a book about “The Future of Science”; this post is part of a series where I try out ideas from the book in an open forum. A summary of many of the themes in the book is available in this essay. If you’d like to be notified when the book is available, please send a blank email to email@example.com with the subject “subscribe book”. I’ll email you to let you know in advance of publication. I will not use your email address for any other purpose! You can subscribe to my blog here.