Author: Michael Nielsen

The penultimate installment in my series. This one is rather short, and something of a placeholder; the final installent is rather more substantive, I promise! It may be a few days before I post the final installment, as I’m heading off to Fraser Island for the weekend, and I may not get it posted before I go.

This one is short in large part because it’s about something most scientists receive a lot of training in: problem-solving, albeit not necessarily of the kind that so often arises in research – the kind where figuring out what the appropriate formulation of the problem is may be half the battle. So I wanted to make a few remarks in the essay about the necessity of finding and keeping forward momentum, to keep the research fog at bay.

One final comment before getting into today’s installment. An unexpected consequence of writing this essay has been the emergence of a theme I neither planned, nor was fully conscious of before starting the essay. That theme is the tension that exists between the short- and the long term. This is not an easy tension to resolve, but I believe it lies at the heart of many difficulties in research.

A quote of Lois Bujold, that I put earlier on the blog, comes to mind again:

All great human deeds both consume and transform their doers. Consider an athlete, or a scientist, or an artist, or an independent business creator. In service of their goals they lay down time and energy and many other choices and pleasures; in return, they become most truly themselves. A false destiny may be spotted by the fact that it consumes without transforming, without giving back the enlarged self.

Oppenheimer spoke in a letter to his brother about the paramount importance of discipline in his life. I didn’t understand this emphasis when I first read it, but I now think that Oppenheimer was speaking of the same tension that I have seen emerge in this essay, and how one must resolve it.

On with today’s installment…

The skills of the problem-solver

As I’ve already said, our technical training as physicists focuses a lot more on problem-solving than problem-creation, so I’m not going to say a lot about the skills needed to be a problem-solver. But I will make a few general remarks that I find helpful.

Clarity, goals, and forward momentum: In my opinion, there is little that is more important in research than building forward momentum. Being clear about some goal, even if that goal is the wrong goal, or the clarity is illusory, is tremendously powerful. For the most part, it’s better to be doing something, rather than nothing, provided, of course, that you set time aside frequently for reflection and reconsideration of your goals. Much of the time in research is spent in a fog, and taking the time to set clear goals can really help lift the fog.

Have multiple formulations: One of the most common mistakes made by researchers is to hold on very closely to a particular problem formulation. They will stick closely to a particular formulation of a problem, without asking if they can achieve insights on related problems. The important thing is to be able to make some progress: if you can find a related problem, or reformulate a problem in a way that permits you to move forward, that is progress.

Spontaneous discovery as the outcome of self-development: For me this is one of the most common ways of making discoveries. Many people’s basic research model is to identify a problem they find interesting, and then spend a lot of time working on just that problem. In fact, if you keep your mind open while engaging in exploration, and are working at the edge of what is known, you’ll often see huge opportunities open wide in front of you, provided you keep developing your range of skills.

No time for an extended post today, but just wanted to quickly mention a great bunch of links pointed out in the comments by Suresh Venkatasubramanian, all on the process of doing research.

First is an interesting post on Suresh’s blog, which in turn links to an amazing repository of expository writing from 1998 Fields Medalist Timothy Gowers, and in particular a thought-provoking essay by Gowers on Two Cultures in mathematical research, with some interesting parallels to the problem creator / problem solver divide I described yesterday.

A longer post than usual today. We’re now getting to the stuff that was in many ways the most fun to write, more concerned with the nitty-gritty of doing research. Today’s post is concerned with different styles of doing research. In particular, I describe two idealizations, the “problem solver” and the “problem creator”, and talk a little about how problem creators work. The next post will be about problem solvers.

The creative process

The problem-solver and the problem-creator

Different people have different styles of creative work. I want to discuss two different styles that I think are particularly useful in understanding the creative process. I call these the problem-solver and the problem-creator styles. They’re not really disjoint or exclusive styles of working, but rather idealizations which are useful ways of thinking about how people go about creative work.

The problem-solver: This is the person who works intensively on well-posed technical problems, often problems known (and sometimes well-known) to the entire research community in which they work. The best problem-solvers are often extremely technically proficient and hard-working. Problem-solvers often attach great social cache to the level of difficulty of the problem they solve, without necessarily worrying so much about other indicators of the importance of the problem.

The problem-creator: This is a rarer working style. Problem-creators may often write papers that are technically rather simple, but ask an interesting new question, or pose an old problem in a new way, or demonstrate a simple but fruitful connection that no-one previously realized existed.

Of course, the problem-solver and the problem-creator are idealizations; all researchers exemplify both styles, to some extent. But they are also useful models to clarify our thinking about the creative process. One distinction between the two styles is how proactive one is in identifying problems, with the problem-solver being much more passive, while the problem-creator is extremely proactive. By contrast, the problem-solver needs to be much more proactive in developing their problem-solving skills. Both styles of research can be extremely successful.

Problem-solvers have numerous social advantages in research, and for that reason I believe they tend to be more common. In particular, it is relatively easy to recognize (and then reward) people who solve problems that are of medium or high levels of difficulty. This has rewards both in terms of the immediate esteem of one’s peers – physicists love to trade legends about brilliant colleagues who immediately see through to the solution of some difficult problems or another – and also in the hunt for jobs and other tangible forms of recognition. It takes more time (and thus can be more difficult) to recognize people whose work is technically rather simple, but whose questions may eventually open up whole new lines of enquiry.

The advantage in being a problem-creator is that there is a sizeable comparative advantage in opening up an entirely new problem area, and thus being the first into that problem area. You can work hard to get a basic foundation in the skills needed in that problem area, and then clean up many of the fundamental problems.

The skills of the problem-creator

Our training as physicists focuses pretty heavily on becoming problem-solvers; we tend not to get much training as problem-creators. One reason I’m discussing these two working styles at some length is to dispel the common idea that creative research is necessarily primarily about problem-solving. It’s true that many people have very successful research career as problem-solvers. But you can also consciously decide to invest more time and effort into developing as a problem-creator. I now describe some of the skills involved in problem-creation.

Developing a taste for what’s important: What do you think are the characteristics of important science? What makes one area thrive, while another dies away? What sorts of unifying ideas are the most useful? What have been the most important developments in your field? Why are they important? What were the apparently promising ideas that didn’t pan out? Why didn’t they pan out? You need to be thinking constantly about these issues, both in concrete terms, and also in the abstract, developing both a general feeling for what is important (and what is not), and also some specific beliefs about what is important and what is not in your fields of interest. Richard Hamming describes setting aside time each week for “Great Thoughts”, time in which he would focus on and discuss with others only things that he believed were of the highest importance. Systematically setting aside time to think (and talk with colleagues) about where the important problems are is an excellent way of developing as a problem-creator.

On this topic, let me point out one myth that exerts a powerful influence (often subconsciously) on people: the idea that difficulty is a good indicator of the importance of a problem. It is true that an elegant solution to a difficult problem (even one not a priori important) often contains important ideas. However, I believe that most people consistently over rate the importance of difficulty. Often far more important is what your work enables, the connections that it makes apparent, the unifying themes uncovered, the new questions asked, and so on.

Internal and external standards for what is important: Some of the most thought-provoking advice on physics that I ever heard was at a colloquium given by eminent physicist Max Dresden. He advised young people in the audience not to work towards a Nobel Prize, but instead to aim their research in directions that they personally find fun and interesting. I thought his advice quite sound in some regards: for some people it is extremely tempting to regard external recognition as the be-all and end-all of research success, and the Nobel Prize is perhaps the highest form of external recognition in physics. Dresden is right, in the sense that working with a primary goal of winning a Nobel Prize would be pointless and degrading; far better to work in an area one personally finds enjoyable.

On the other hand, the Nobel Prizes are usually given for very good reasons: they reward some of the most interesting work in all of physics. There is, admittedly, a political element, with certain fields being favoured, and so on. Nonetheless, imagine a world in which one of these discoveries had not been awarded a Prize for some reason. Would you be proud to have your name associated with that discovery, even so, and regard the work on it as time well spent? In every case I can think of, that certainly is the case for me, and I suspect it’s true for most other physicists.

I believe this highlights an interesting point about what makes something interesting and important. A person working toward a Nobel Prize or some other form of external recognition has, in some sense, decided to abdicate their personal decision about what is important and interesting. The external community of physicists (in this case, represented by the Nobel Committee) is what makes their decision: if it might win a Nobel, it’s important.

Balancing this observation, this is not to say that your decision about what is interesting and important should be yours along. People who work in isolation rarely end up making contributions that are all that significant. Your decision about what is important should be informed by others: talk to your peers, find out what they think is important, look in the textbooks and history books and biographies, and, yes, look at what wins prizes (of all sorts).

But at the end of the day you’ve got to form your own independent standards for what is interesting and important and worth doing, and make judgments about where you should be making a contribution, based on those standards. I think better advice from Dresden would have been to aim to produce work of the highest possible caliber, but according to what you have come to believe is important.

Exploring for problems: Obviously, all researchers do some of this. For the problem-solver, the process of exploring for problems often works along the following lines: keep moving around, looking for problems that you consider (a) well-posed, or able to be well-posed after some work on your part, (b) likely to fall within a reasonable time to the arsenal of tools at your disposal (perhaps with some small expansion of that arsenal), and (c) below some minimum thresholds of interest and difficulty. Once you’ve found a problem of this sort, you work hard on the problem, solve it, and publish.

Problem-creators may be rather more systematic about exploring for problems. For example, they may occasionally set time aside to survey the landscape of a field, looking not just for problems, but trying to identify larger patterns. What types of questions do people in the field tend to ask? Can we abstract away patterns in those questions? What other fields might there be links to? What are the few most important problems in the field? Problem-creators set aside time for doing this kind of systematic exploration, and do it in a disciplined way, often with feedback from others.

Surveying the landscape can be particularly revealing. A lot of people work in fashionable subfields of a larger field primarily because there are lots of other people working in that subfield. The problems they work on may be technically complicated, especially after a few years, when the most basic questions have been answered. This is compensated by the fact that it’s extremely comforting to work within a field where there is a standard narrative explaining the importance of the field, some canonical models for what problems are interesting, and a willing audience of people ready to appreciate your work. In addition, working in such subfields gives younger people a chance to show off their technical prowess (sometimes, not unlike elk spoiling for a fight) to peers in a position to recommend them for valuable faculty positions.

Getting ahead of the game: There are many important problems, and sometimes an entire field comes to some agreement about what is important: proving the Riemann Hypothesis, or understanding high temperature superconductivity. Sometimes, however, there is a problem either not appreciated at all, or only dimly appreciated, that is equal in importance to such gems. Consider the creation of the scanning tunneling microscope – the basic idea had been around for years, yet nobody had ever seriously tried to build the device. The inventors put it together on a shoestring, and created one of the major tools of modern physics. Or consider David Deutsch and Richard Feynman’s creation of the field of quantum computing, by framing the right questions (“What would a quantum mechanical computer be capable of?” and “Would it be faster than a classical computer?”). One of the big ways you can get ahead as a researcher is by identifying and then solving problems that are important, but perhaps not terribly difficult, ahead of everyone else.

Identify the messes: In a nice article about how he does research, physicist Steven Weinberg emphasized the importance of identifying the messes. What areas of physics appear to be a state of mess? Funnily enough, one of the signs of this can be that it’s very hard to understand. For a long time – and to some extent this persists today – physics texts on general relativity were very difficult to understand. The tensor calculus in them was often confusing and difficult to understand. There was a good reason for this: the basic definitions in the subject of differential geometry, although laid down in the 19th century, didn’t really reach their modern form until the mid part of the twentieth century, and then took considerable time to migrate to physics. The reason a lot of the discussion of tensor calculus in physics texts is confusing is because, very often, it is confused, being written by people who don’t have quite the right definitions (meaning, in this case, simplest, most elegant and natural) in mind.

When you identify such a mess, the natural inclination of many people is to shy away, to find something that is easier to understand. But a field that is a mess is really an opportunity. Chances are good that there are deep unifying and simplifying concepts still waiting to be understood and developed by someone – perhaps you.

About a year ago I gave an hour-long seminar on “Extreme Thinking” to a lay audience at a conference on “Tough Learning”. I thought the essay might be of interest to people who enjoy things like my current series on principles of effective research, and so I’m reposting it.

The “Extreme Thinking” essay differs in two important ways from my ongoing series: (a) it is intended for an audience of non-scientists, and thus needs to be very general; and (b) it is a finished product, and thus is more polished.

Like the Principles series, much of what I say in the present essay is obvious and well-known. Nonetheless, I’m convinved that many of my personal difficulties (and those of others) in doing research come from not doing the obvious things correctly, thus this essay.

Essay: HTML, ASCII, PDF, Microsoft Word.

Presentation: PDF, Powerpoint.

Lance Fortnow has an interesting post on why conferences and conference proceedings are so much more important in CS than in older fields like physics.

Lance’s post started me thinking about a topic I wonder about from time to time: the role preprints play in science. At the moment, the main role journals seem to play in physics is resume-building: for grants, hiring and promotion it’s important to get into the so-called “best” journals, like Science, Nature, and Physical Review Letters. Peer review is often mentioned as another function of journals, but there are other ways that can be accomplished (see, e.g. Daniel Gottesman’s sadly defunct preprint review site.) Resume building does not seem to me to be a sufficient justification for the incredible amount of money spent on journals.

Here’s the next installment in my ongoing series. This one is a bit more of a placeholder than many of the others, so I’ll make a few extra comments before getting into the essay proper.

The main thing is that I believe people consistently underestimate (a) the extent to which having a good research environment helps, and (b) the ability they have to create such an environment.

Related to topic (a), in Malcolm Gladwell’s entertaining book “The Tipping Point”, Gladwell describes a psych experiment in which people were shown videos of different basketball players, and asked to evaluate the ability of those different basketballers. They were additionally told that some of the basketballers were playing in very poor lighting, while others were playing in excellent lighting, and this should be taken into account. Despite this, they consistently rated the basketballers playing in poor lighting as having less ability, despite the fact that the basketballers had actually been chosen to ensure that both groups had equal ability.

Gladwell cites this as one of many studies showing that people consistently underrate the importance of environmental effects, including the effect of their personal environment. Now, you can debate the validity of the basketball study, but I think the key point is highly plausible: people don’t make good evaluations of the contributions environmental factors make to their (or others’) performance.

On topic (b), as an extreme example, I’ve seen grad students essentially start their own research groups, taking on other grad students in a supervisory role, finding space, running seminar series, discussion series and so on, all without direct faculty support. The story of the “dynamical systems collective” at UC Santa Cruz, told in James Gleick’s “Chaos”, is instructive: a bunch of grad students went off and wrote a whole bunch of seminal papers on chaos, all on their own initiative. This might seem like a freak occurrence, but I’ve seen this kind of thing up close: it’s a function of the drive and determination of the person involved, not a freak accident.

With that as prelude, here’s the next installment of the essay:

Develop a high-quality research environment

There is a considerable amount of research showing that people consistently underestimate the effect of the environment on personal effectiveness. This is particularly important in an academic environment where there are usually many short-term social pressures that are not directly related to research effectiveness – teaching, writing letters of recommendation and referee reports, committee work, academic politics. By contrast, in most institutions there are few short-term social pressures to do great research work.

Some of the highest-leverage work you can do involves improving your environment so that social pressures work for you as a researcher, rather than against you. Discussing this in detail would require another essay of length at least equal to that of the present one, but I will make a few remarks.

The first is that improving your environment is something anyone can do; students, in particular, often underestimate the magnitude of the changes they can bring about. Anyone can start a seminar series, develop a discussion area, create a lounge, organize a small workshop, or organize a reading group. Furthermore, although all these things are hard to do well, if you’re willing to do critical evaluations, experiment and try radical changes, preferably in partnership with equally committed people, things are likely to improve a great deal.

Second, institutions have long memories, so changes that you make in your environment will stick around for a long time. This means that once something is working well, chances are it’ll continue to work well without much help from you – and you can move on to improve some other aspect of your environment. Furthermore, each positive change you make actually improves your leverage with other people. I’ve known undergraduate students who had made so many creative positive contributions to their departments that their influence with canny senior faculty was comparable to the influence of other senior faculty.

You can now get your Erdos number down to two, which is rather better than the earlier auction run by William Tozier. (Via Ike Chuang).

Daniel Lemire, quoting Phil Agre, writes a nice post on academic freedom:

It seems you can either build an empire for the purpose of building an empire, because that’s your definition of success, or else, you can aim to remain “free”. That’s a very powerful idea:

You build networks around the issues you care about, you grow and change through the relationships that result, you articulate the themes that are emerging in the community’s work, and through community-building and leadership you get the resources to do the things that you most care about doing. It’s true that this method will never give you arbitrary power. But the desire for arbitrary power is not freedom – it is a particularly abject form of slavery. If you can let go of preconceived plans then you are free: you can choose whom to associate with, and as you build your network you multiply the further directions that you can choose to go. You also multiply the unexpected opportunities that open up, the places you can turn for assistance with your projects, the flows of useful information that keep you in contact with reality, the surveillance of the horizon that keeps you from getting cornered by unanticipated developments, and the public persona that ensures that people keep coming to you with offers that you can take or leave. That is what freedom is, and it is yours if you will do the work.

I give Agre a lot of credit from bringing in the concept of “freedom” in research. University professors will often talk about “academic freedom”. I think that freedom in research is a stronger form of freedom. You can have “academic freedom” but be a slave to the “publish-or-perish” paradigm for the power it brings you. Or else, you can “do the work”, that is, do your research as a network node, and leverage the strength of the network to make the research you want to do anyhow, much better, much stronger.

This, of course, is not the usual concept of academic freedom – the ability to speak truth to power, without losing your job. But it’s an important adjunct; if you need to give up your own freedom in an attempt to maintain power, then that is not freedom. It’s become corny, but I thought the old song had it about right – freedom can be just another word for nothin’ left to lose.

Via – d – I discover Edsger Dijkstra’s three golden rules for scientific research. It’s all interesting, but I particularly liked the first rule:

Raise your quality standards as high as you can live with, avoid wasting your time on routine roblems, and always try to work as closely as possible at the boundary of your abilities. Do this, because it is the only way of discovering how that boundary should be moved forward.

There’s a lot of truth in that.

The pedant in me can’t resist quibbling, at least a little: if you haven’t had a lot of success recently (or ever), spending some time on routine problems can help raise your morale, ensure you stay or come into contact with your community of researchers, and – most important – means that you earn your keep.

Developing research strengths

The foundation is a plan for the development of research strengths. What are you interested in? Given your interests, what are you going to try to learn? The plan needs to be driven by your research goals, but should balance short-term and long-term considerations. Some time should be spent on things that appear very likely to lead to short-term research payoff. Equally well, some time needs to be allocated to the development of strengths that may not have much immediate pay-off, but over the longer-term will have a considerable payoff.

In targeting areas of development, an important goal to keep in mind is that you want to develop unique combinations of abilities. You need to develop unique combinations of talents which give you a comparative advantage over other people. Do what you can do better than anybody; to mangle a quote from Lincoln, nobody can be better than everybody all of the time, but anybody can be better than everybody some of the time.

In my opinion the reason most people fail to do great research is that they are not willing to pay the price in self-development. Say some new field opens up that combines field X and field Y. Researchers from each of these fields flock to the new field. My experience is that virtually none of the researchers in either field will systematically learn the other field in any sort of depth. The few who do put in this effort often achieve spectacular results.

Finally, a note on how to go about developing some new research strength. A mistake I’m prone to make, and I know some others are as well, is to feel as though some degree of completeness is required in understanding a research field. In fact, in any given research field there are usually only a tiny number of papers that are really worth reading. You are almost certainly better off reading deeply in the ten most important papers of a research field than you are skimming the top five hundred.

These ideas carry over to the problem of staying current in your fields of interest: I believe that you can stay quite current by (a) quickly skimming a great deal of work, to keep track of what is known, and what sort of problems people are thinking about, and (b) based on that skimming, picking a dozen or so papers each year to read deeply, in the belief that they contain the most important research results of the year. This is not the only deep reading you’ll need to do; you’ll also need to do some which is related to the immediate problems that you’re working on. But you certainly should do some such deep reading.