Five problems with doing research in the open
I’m an advocate of extreme openness in doing research. I believe online tools are greatly underused by scientists, and that taking a far more open approach to the sharing of scientific ideas and data is one important step to taking full advantage of those tools.
Although there are great advantages to such openness, there are also many problems. The biggest problems, which I’ve talked about here before, are to do with lack of incentives to share information, and the relatively undeveloped state of online tools. While tools like wikis and blogs are great for some purposes, they’re just not all that well adapted to research.
This post enumerates some of the other problems with extreme openness. None are insurmountable, and none change my belief that extreme openness will be extremely valuable to the progress of science. But they seem worth writing out systematically. In the interests of brevity I’ve concentrated on listing problems, not proposing solutions, although solutions to many come readily to mind.
No-one wants someone looking over their shoulder while they work: As I draft my book, there’s times when I’m best off working away in private, without interruption and distraction, and times when I could really do with some scrutiny. It’s plain irritating and disheartening to be flamed by someone for a flaw in your work that you’re already aware of, and planning to fix. A partial solution is to do research in a low-visibility but essentially open fashion. Many open source projects do this – work on the project is carried out via open but obscure channels (e.g., mailing lists), and then major releases are announced in a high visibility channel.
Conversation doesn’t scale naively: In an ideal world, online tools will connect people with well-matched and complementary expertise. What you see instead in a lot of online conversations is experts “connected” to the rude and uninformed. I don’t know that this brings much benefit to anyone. I think this problem can be solved with improved design of the tools, and improved filtering, but it’s a real problem.
Groupthink: Original thinking that goes against conventional wisdom is not always well rewarded in the open. Commentary is often knee-jerk, rather than constructive. As an example, one of the most original thinkers I know is Robin Hanson; Robin often blogs  interesting ideas that contradict conventional wisdom. There’s some great comments on his blog posts, but there’s also a lot of noise, and poorly thought-out responses as people see their sacred cows challenged.
Giving offence: This is really an instance of the previous problem: it’s difficult to develop ideas that may give offense in the open. I don’t think I’d want to write a research article about gun control out in the open.
Ethics and IP concerns: Sometimes, confidentiality may prevent disclosure of data, either for ethical reasons or to protect IP, or both.
 Robin is only one of many contributors to that blog, and you may need to scroll down to find some of Robin’s contributions.