This is the final installment in my series on the principles of effective research.

This installment was, in some ways, the most fun to write: it’s some thoughts on the difficulties involved in solving the most important research problems, and how to address those difficulties.

Now, I’ve had less success than I would like in this regard (I doubt I’m alone), so this should all be taken with a big grain of salt. Nonetheless, like so many other research scientists, I’d like to solve some big problems, and this post is an attempt to describe a bit of a model for how to go about doing so.

Enjoy!

Working on important problems

It’s important that you work towards being able to solve important problems. This sounds silly, but people don’t do this for any number of reasons. I want to talk a little about those reasons, and how to avoid them.

Reason 1: Lack of self-development. Many people don’t spend enough time on self-development. If you stop your development at the level which resulted in your first paper, it’s unlikely you’ll solve any major problems. More realistically, for many people self-development is an incidental thing, something that happens while they’re on the treadmill of trying to solve problems, generate papers, and so on, or while teaching. While such people will develop, it’s unlikely that doing so in such an ad hoc way will let them address the most important problems.

Reason 2: The treadmill of small problems. Social factors such as the need to publish, get grants, and so on, encourage people to work only on unimportant problems, without addressing the important problems. This can be a difficult treadmill to get off.

My belief is that the way to start out in a research career is by working primarily on small and relatively tractable problems, where you have a good chance of success. You then continue the process of self-development, gradually working up to more important problems (which also tend to be more difficult, although, as noted above, difficulty is most emphatically not the same as importance). The rare exception is important problems that are also likely to be technically easy; if you’re lucky you may find such a problem early in your career, or be handed one. If so, solve it quickly!

Even later on, when you’ve developed to the point that you can realistically expect to be able to attack important problems, it’s still useful to tackle a mixture of more and less important problems. The reason is that tackling smaller problems ensures that you make a reasonable contribution to science, and that you continue to take an active part in the research community. Even Andrew Wiles continued to publish papers and work on other problems during his work on Fermat’s Last Theorem, albeit at a rather low rate. If he had not, he would have lost contact with an entire research community, and losing such contact would likely have made a significant negative difference to his work on Fermat’s Last Theorem.

Reason 3: The intimidation factor. Even if people have spent enough time on self-development that they have a realistic chance of attacking big problems, they still may not. The reason is that they have a fear of working on something unsuccessfully. Imagine Andrew Wiles feeling if he had worked on Fermat’s Last Theorem for several decades, and completely failed. For most people, the fear of ending up in such a situation is enough to discourage them from doing this.

The great mathematician Andrei Kolmogorov described an interesting trick that he used to get around this problem. Rather than investing all his time and effort on attacking the problem, he’d put the problem into a larger context. He’d announce a seminar series in which he’d lecture on material that he thought would be related to the problem. He’d write a set of lecture notes (often turning into a book) on material related to the problem. That way, he lowered the psychological pressure on himself. Rather than investing all his effort in an attack on the problem – which might ultimately be a complete waste of time – he knew that he’d produce something of value. By making the research process part of a larger endeavour, he ensured that the process was a success no matter how it came out, even if he failed to solve the problem, or was scooped by someone else. It’s a case of not putting all of one’s psychological eggs in one basket.

Richard Feynman described a related trick. He said that when he started working on a problem he would try to convince himself that he had some kooky insight into the problem that it was very unlikely anybody else had. He admitted that very often this belief was erroneous, or that, even if original, his initial insight often wasn’t very good. But he claimed that he found that he could fool himself into thinking that he had the “inside track” on the problem as a result, and this was essential to getting up the forward momentum necessary to really make a big dint in a difficult problem.

Committing to work on an important problem: For the difficult problems, I think commitment is really a process rather than a moment. You may decide to prepare a lecture to talk about a problem. If that is interesting, you enjoy it, and you feel like you have some insight, you might decide to prepare a few lectures. If that goes well, perhaps you’ll start to prepare more lectures, write a review, and maybe make a really big contribution. It’s all about building up more and more insight. Ideally, you’ll do this as part of some larger process, with social support around you.

People who only attack difficult problems: There is a converse to the problem I’ve been talking about, which is people who are only interested in attacking problems that are both difficult and important. This affliction can affect people at any stage of their career, but it manifests itself in somewhat different ways at different stages.

In the case of the beginner, this is like a beginning pole vaulter insisting on putting the bar at 5 meters from the time they begin, rather than starting at some more reasonable height. Unless exceptionally pigheaded, such a person will never learn to vault 5 meters successfully, simply because they will never learn anything from failure at a more realistic starting height. This sounds
prima facie ridiculous, but I have seen people burn out by following exactly this strategy.

The case of the more experienced researcher is more difficult. As I’ve emphasized, once you’ve reached an appropriate level of development I think it’s important to spend some time working on the most important problems. But if that’s all you do, there are some very significant drawbacks. In particular, by attacking only the most important and most difficult problems an experienced researcher (a) takes themselves out of circulation, (b) stops making ongoing contributions, (c) loses the habit of success, and (d) risks losing morale, which is so important to research success. I think the solution is to balance one’s work on the more and less important problems: you need to schedule time to do the more important stuff, but should also make sure that you spend some time on less high-risk activities.

In both cases, the explanation is often, at least in part, intellectual macho. Theorists can be a pretty judgmental lot about the value of their own work, and the work of others. This helps lead some into the error of only working on big problems, and not sometimes working on little problems, for the fun of it, for the contact it brings with colleagues, and for the rewarding and enjoyable sense of making a real contribution of some significance.

Stephen Hawking’s disavowal of his belief that black holes destroy information has received a lot of coverage, both in the press and the blogosphere.

I don’t have a lot to add over what has been said elsewhere, especially Sean Caroll’s recent series of posts (including a transcript of Hawking’s talk) on his blog. But I will point to John Preskill’s nice discussion of what was at stake.

A little advertising: on Thursday September 23 and Friday September 24 the University of Queensland Physics Department is going to be holding a Postgraduate Information and Recruitment Day. Australian and New Zealand students who might be interested in doing a PhD in physics are strongly encouraged to apply. See the above links for more information about research activities within the Department.

Successful applicants will be flown to Brisbane to participate in activities on Thursday afternoon and Friday; we will also provide accommodation for Saturday and Sunday, if you should wish to stay and explore Brisbane and surrounds. I think the event last year was a great deal of fun for all concerned, and am looking forward to it again this year.

The penultimate installment in my series. This one is rather short, and something of a placeholder; the final installent is rather more substantive, I promise! It may be a few days before I post the final installment, as I’m heading off to Fraser Island for the weekend, and I may not get it posted before I go.

This one is short in large part because it’s about something most scientists receive a lot of training in: problem-solving, albeit not necessarily of the kind that so often arises in research – the kind where figuring out what the appropriate formulation of the problem is may be half the battle. So I wanted to make a few remarks in the essay about the necessity of finding and keeping forward momentum, to keep the research fog at bay.

One final comment before getting into today’s installment. An unexpected consequence of writing this essay has been the emergence of a theme I neither planned, nor was fully conscious of before starting the essay. That theme is the tension that exists between the short- and the long term. This is not an easy tension to resolve, but I believe it lies at the heart of many difficulties in research.

A quote of Lois Bujold, that I put earlier on the blog, comes to mind again:

All great human deeds both consume and transform their doers. Consider an athlete, or a scientist, or an artist, or an independent business creator. In service of their goals they lay down time and energy and many other choices and pleasures; in return, they become most truly themselves. A false destiny may be spotted by the fact that it consumes without transforming, without giving back the enlarged self.

Oppenheimer spoke in a letter to his brother about the paramount importance of discipline in his life. I didn’t understand this emphasis when I first read it, but I now think that Oppenheimer was speaking of the same tension that I have seen emerge in this essay, and how one must resolve it.

On with today’s installment…

The skills of the problem-solver

As I’ve already said, our technical training as physicists focuses a lot more on problem-solving than problem-creation, so I’m not going to say a lot about the skills needed to be a problem-solver. But I will make a few general remarks that I find helpful.

Clarity, goals, and forward momentum: In my opinion, there is little that is more important in research than building forward momentum. Being clear about some goal, even if that goal is the wrong goal, or the clarity is illusory, is tremendously powerful. For the most part, it’s better to be doing something, rather than nothing, provided, of course, that you set time aside frequently for reflection and reconsideration of your goals. Much of the time in research is spent in a fog, and taking the time to set clear goals can really help lift the fog.

Have multiple formulations: One of the most common mistakes made by researchers is to hold on very closely to a particular problem formulation. They will stick closely to a particular formulation of a problem, without asking if they can achieve insights on related problems. The important thing is to be able to make some progress: if you can find a related problem, or reformulate a problem in a way that permits you to move forward, that is progress.

Spontaneous discovery as the outcome of self-development: For me this is one of the most common ways of making discoveries. Many people’s basic research model is to identify a problem they find interesting, and then spend a lot of time working on just that problem. In fact, if you keep your mind open while engaging in exploration, and are working at the edge of what is known, you’ll often see huge opportunities open wide in front of you, provided you keep developing your range of skills.

No time for an extended post today, but just wanted to quickly mention a great bunch of links pointed out in the comments by Suresh Venkatasubramanian, all on the process of doing research.

First is an interesting post on Suresh’s blog, which in turn links to an amazing repository of expository writing from 1998 Fields Medalist Timothy Gowers, and in particular a thought-provoking essay by Gowers on Two Cultures in mathematical research, with some interesting parallels to the problem creator / problem solver divide I described yesterday.

A longer post than usual today. We’re now getting to the stuff that was in many ways the most fun to write, more concerned with the nitty-gritty of doing research. Today’s post is concerned with different styles of doing research. In particular, I describe two idealizations, the “problem solver” and the “problem creator”, and talk a little about how problem creators work. The next post will be about problem solvers.

The creative process

The problem-solver and the problem-creator

Different people have different styles of creative work. I want to discuss two different styles that I think are particularly useful in understanding the creative process. I call these the problem-solver and the problem-creator styles. They’re not really disjoint or exclusive styles of working, but rather idealizations which are useful ways of thinking about how people go about creative work.

The problem-solver: This is the person who works intensively on well-posed technical problems, often problems known (and sometimes well-known) to the entire research community in which they work. The best problem-solvers are often extremely technically proficient and hard-working. Problem-solvers often attach great social cache to the level of difficulty of the problem they solve, without necessarily worrying so much about other indicators of the importance of the problem.

The problem-creator: This is a rarer working style. Problem-creators may often write papers that are technically rather simple, but ask an interesting new question, or pose an old problem in a new way, or demonstrate a simple but fruitful connection that no-one previously realized existed.

Of course, the problem-solver and the problem-creator are idealizations; all researchers exemplify both styles, to some extent. But they are also useful models to clarify our thinking about the creative process. One distinction between the two styles is how proactive one is in identifying problems, with the problem-solver being much more passive, while the problem-creator is extremely proactive. By contrast, the problem-solver needs to be much more proactive in developing their problem-solving skills. Both styles of research can be extremely successful.

Problem-solvers have numerous social advantages in research, and for that reason I believe they tend to be more common. In particular, it is relatively easy to recognize (and then reward) people who solve problems that are of medium or high levels of difficulty. This has rewards both in terms of the immediate esteem of one’s peers – physicists love to trade legends about brilliant colleagues who immediately see through to the solution of some difficult problems or another – and also in the hunt for jobs and other tangible forms of recognition. It takes more time (and thus can be more difficult) to recognize people whose work is technically rather simple, but whose questions may eventually open up whole new lines of enquiry.

The advantage in being a problem-creator is that there is a sizeable comparative advantage in opening up an entirely new problem area, and thus being the first into that problem area. You can work hard to get a basic foundation in the skills needed in that problem area, and then clean up many of the fundamental problems.

The skills of the problem-creator

Our training as physicists focuses pretty heavily on becoming problem-solvers; we tend not to get much training as problem-creators. One reason I’m discussing these two working styles at some length is to dispel the common idea that creative research is necessarily primarily about problem-solving. It’s true that many people have very successful research career as problem-solvers. But you can also consciously decide to invest more time and effort into developing as a problem-creator. I now describe some of the skills involved in problem-creation.

Developing a taste for what’s important: What do you think are the characteristics of important science? What makes one area thrive, while another dies away? What sorts of unifying ideas are the most useful? What have been the most important developments in your field? Why are they important? What were the apparently promising ideas that didn’t pan out? Why didn’t they pan out? You need to be thinking constantly about these issues, both in concrete terms, and also in the abstract, developing both a general feeling for what is important (and what is not), and also some specific beliefs about what is important and what is not in your fields of interest. Richard Hamming describes setting aside time each week for “Great Thoughts”, time in which he would focus on and discuss with others only things that he believed were of the highest importance. Systematically setting aside time to think (and talk with colleagues) about where the important problems are is an excellent way of developing as a problem-creator.

On this topic, let me point out one myth that exerts a powerful influence (often subconsciously) on people: the idea that difficulty is a good indicator of the importance of a problem. It is true that an elegant solution to a difficult problem (even one not a priori important) often contains important ideas. However, I believe that most people consistently over rate the importance of difficulty. Often far more important is what your work enables, the connections that it makes apparent, the unifying themes uncovered, the new questions asked, and so on.

Internal and external standards for what is important: Some of the most thought-provoking advice on physics that I ever heard was at a colloquium given by eminent physicist Max Dresden. He advised young people in the audience not to work towards a Nobel Prize, but instead to aim their research in directions that they personally find fun and interesting. I thought his advice quite sound in some regards: for some people it is extremely tempting to regard external recognition as the be-all and end-all of research success, and the Nobel Prize is perhaps the highest form of external recognition in physics. Dresden is right, in the sense that working with a primary goal of winning a Nobel Prize would be pointless and degrading; far better to work in an area one personally finds enjoyable.

On the other hand, the Nobel Prizes are usually given for very good reasons: they reward some of the most interesting work in all of physics. There is, admittedly, a political element, with certain fields being favoured, and so on. Nonetheless, imagine a world in which one of these discoveries had not been awarded a Prize for some reason. Would you be proud to have your name associated with that discovery, even so, and regard the work on it as time well spent? In every case I can think of, that certainly is the case for me, and I suspect it’s true for most other physicists.

I believe this highlights an interesting point about what makes something interesting and important. A person working toward a Nobel Prize or some other form of external recognition has, in some sense, decided to abdicate their personal decision about what is important and interesting. The external community of physicists (in this case, represented by the Nobel Committee) is what makes their decision: if it might win a Nobel, it’s important.

Balancing this observation, this is not to say that your decision about what is interesting and important should be yours along. People who work in isolation rarely end up making contributions that are all that significant. Your decision about what is important should be informed by others: talk to your peers, find out what they think is important, look in the textbooks and history books and biographies, and, yes, look at what wins prizes (of all sorts).

But at the end of the day you’ve got to form your own independent standards for what is interesting and important and worth doing, and make judgments about where you should be making a contribution, based on those standards. I think better advice from Dresden would have been to aim to produce work of the highest possible caliber, but according to what you have come to believe is important.

Exploring for problems: Obviously, all researchers do some of this. For the problem-solver, the process of exploring for problems often works along the following lines: keep moving around, looking for problems that you consider (a) well-posed, or able to be well-posed after some work on your part, (b) likely to fall within a reasonable time to the arsenal of tools at your disposal (perhaps with some small expansion of that arsenal), and (c) below some minimum thresholds of interest and difficulty. Once you’ve found a problem of this sort, you work hard on the problem, solve it, and publish.

Problem-creators may be rather more systematic about exploring for problems. For example, they may occasionally set time aside to survey the landscape of a field, looking not just for problems, but trying to identify larger patterns. What types of questions do people in the field tend to ask? Can we abstract away patterns in those questions? What other fields might there be links to? What are the few most important problems in the field? Problem-creators set aside time for doing this kind of systematic exploration, and do it in a disciplined way, often with feedback from others.

Surveying the landscape can be particularly revealing. A lot of people work in fashionable subfields of a larger field primarily because there are lots of other people working in that subfield. The problems they work on may be technically complicated, especially after a few years, when the most basic questions have been answered. This is compensated by the fact that it’s extremely comforting to work within a field where there is a standard narrative explaining the importance of the field, some canonical models for what problems are interesting, and a willing audience of people ready to appreciate your work. In addition, working in such subfields gives younger people a chance to show off their technical prowess (sometimes, not unlike elk spoiling for a fight) to peers in a position to recommend them for valuable faculty positions.

Getting ahead of the game: There are many important problems, and sometimes an entire field comes to some agreement about what is important: proving the Riemann Hypothesis, or understanding high temperature superconductivity. Sometimes, however, there is a problem either not appreciated at all, or only dimly appreciated, that is equal in importance to such gems. Consider the creation of the scanning tunneling microscope – the basic idea had been around for years, yet nobody had ever seriously tried to build the device. The inventors put it together on a shoestring, and created one of the major tools of modern physics. Or consider David Deutsch and Richard Feynman’s creation of the field of quantum computing, by framing the right questions (“What would a quantum mechanical computer be capable of?” and “Would it be faster than a classical computer?”). One of the big ways you can get ahead as a researcher is by identifying and then solving problems that are important, but perhaps not terribly difficult, ahead of everyone else.

Identify the messes: In a nice article about how he does research, physicist Steven Weinberg emphasized the importance of identifying the messes. What areas of physics appear to be a state of mess? Funnily enough, one of the signs of this can be that it’s very hard to understand. For a long time – and to some extent this persists today – physics texts on general relativity were very difficult to understand. The tensor calculus in them was often confusing and difficult to understand. There was a good reason for this: the basic definitions in the subject of differential geometry, although laid down in the 19th century, didn’t really reach their modern form until the mid part of the twentieth century, and then took considerable time to migrate to physics. The reason a lot of the discussion of tensor calculus in physics texts is confusing is because, very often, it is confused, being written by people who don’t have quite the right definitions (meaning, in this case, simplest, most elegant and natural) in mind.

When you identify such a mess, the natural inclination of many people is to shy away, to find something that is easier to understand. But a field that is a mess is really an opportunity. Chances are good that there are deep unifying and simplifying concepts still waiting to be understood and developed by someone – perhaps you.

About a year ago I gave an hour-long seminar on “Extreme Thinking” to a lay audience at a conference on “Tough Learning”. I thought the essay might be of interest to people who enjoy things like my current series on principles of effective research, and so I’m reposting it.

The “Extreme Thinking” essay differs in two important ways from my ongoing series: (a) it is intended for an audience of non-scientists, and thus needs to be very general; and (b) it is a finished product, and thus is more polished.

Like the Principles series, much of what I say in the present essay is obvious and well-known. Nonetheless, I’m convinved that many of my personal difficulties (and those of others) in doing research come from not doing the obvious things correctly, thus this essay.

Essay: HTML, ASCII, PDF, Microsoft Word.

Presentation: PDF, Powerpoint.

Lance Fortnow has an interesting post on why conferences and conference proceedings are so much more important in CS than in older fields like physics.

Lance’s post started me thinking about a topic I wonder about from time to time: the role preprints play in science. At the moment, the main role journals seem to play in physics is resume-building: for grants, hiring and promotion it’s important to get into the so-called “best” journals, like Science, Nature, and Physical Review Letters. Peer review is often mentioned as another function of journals, but there are other ways that can be accomplished (see, e.g. Daniel Gottesman’s sadly defunct preprint review site.) Resume building does not seem to me to be a sufficient justification for the incredible amount of money spent on journals.

Here’s the next installment in my ongoing series. This one is a bit more of a placeholder than many of the others, so I’ll make a few extra comments before getting into the essay proper.

The main thing is that I believe people consistently underestimate (a) the extent to which having a good research environment helps, and (b) the ability they have to create such an environment.

Related to topic (a), in Malcolm Gladwell’s entertaining book “The Tipping Point”, Gladwell describes a psych experiment in which people were shown videos of different basketball players, and asked to evaluate the ability of those different basketballers. They were additionally told that some of the basketballers were playing in very poor lighting, while others were playing in excellent lighting, and this should be taken into account. Despite this, they consistently rated the basketballers playing in poor lighting as having less ability, despite the fact that the basketballers had actually been chosen to ensure that both groups had equal ability.

Gladwell cites this as one of many studies showing that people consistently underrate the importance of environmental effects, including the effect of their personal environment. Now, you can debate the validity of the basketball study, but I think the key point is highly plausible: people don’t make good evaluations of the contributions environmental factors make to their (or others’) performance.

On topic (b), as an extreme example, I’ve seen grad students essentially start their own research groups, taking on other grad students in a supervisory role, finding space, running seminar series, discussion series and so on, all without direct faculty support. The story of the “dynamical systems collective” at UC Santa Cruz, told in James Gleick’s “Chaos”, is instructive: a bunch of grad students went off and wrote a whole bunch of seminal papers on chaos, all on their own initiative. This might seem like a freak occurrence, but I’ve seen this kind of thing up close: it’s a function of the drive and determination of the person involved, not a freak accident.

With that as prelude, here’s the next installment of the essay:

Develop a high-quality research environment

There is a considerable amount of research showing that people consistently underestimate the effect of the environment on personal effectiveness. This is particularly important in an academic environment where there are usually many short-term social pressures that are not directly related to research effectiveness – teaching, writing letters of recommendation and referee reports, committee work, academic politics. By contrast, in most institutions there are few short-term social pressures to do great research work.

Some of the highest-leverage work you can do involves improving your environment so that social pressures work for you as a researcher, rather than against you. Discussing this in detail would require another essay of length at least equal to that of the present one, but I will make a few remarks.

The first is that improving your environment is something anyone can do; students, in particular, often underestimate the magnitude of the changes they can bring about. Anyone can start a seminar series, develop a discussion area, create a lounge, organize a small workshop, or organize a reading group. Furthermore, although all these things are hard to do well, if you’re willing to do critical evaluations, experiment and try radical changes, preferably in partnership with equally committed people, things are likely to improve a great deal.

Second, institutions have long memories, so changes that you make in your environment will stick around for a long time. This means that once something is working well, chances are it’ll continue to work well without much help from you – and you can move on to improve some other aspect of your environment. Furthermore, each positive change you make actually improves your leverage with other people. I’ve known undergraduate students who had made so many creative positive contributions to their departments that their influence with canny senior faculty was comparable to the influence of other senior faculty.

You can now get your Erdos number down to two, which is rather better than the earlier auction run by William Tozier. (Via Ike Chuang).