Month: July 2004

Here’s the next installment in my ongoing series. This one is a bit more of a placeholder than many of the others, so I’ll make a few extra comments before getting into the essay proper.

The main thing is that I believe people consistently underestimate (a) the extent to which having a good research environment helps, and (b) the ability they have to create such an environment.

Related to topic (a), in Malcolm Gladwell’s entertaining book “The Tipping Point”, Gladwell describes a psych experiment in which people were shown videos of different basketball players, and asked to evaluate the ability of those different basketballers. They were additionally told that some of the basketballers were playing in very poor lighting, while others were playing in excellent lighting, and this should be taken into account. Despite this, they consistently rated the basketballers playing in poor lighting as having less ability, despite the fact that the basketballers had actually been chosen to ensure that both groups had equal ability.

Gladwell cites this as one of many studies showing that people consistently underrate the importance of environmental effects, including the effect of their personal environment. Now, you can debate the validity of the basketball study, but I think the key point is highly plausible: people don’t make good evaluations of the contributions environmental factors make to their (or others’) performance.

On topic (b), as an extreme example, I’ve seen grad students essentially start their own research groups, taking on other grad students in a supervisory role, finding space, running seminar series, discussion series and so on, all without direct faculty support. The story of the “dynamical systems collective” at UC Santa Cruz, told in James Gleick’s “Chaos”, is instructive: a bunch of grad students went off and wrote a whole bunch of seminal papers on chaos, all on their own initiative. This might seem like a freak occurrence, but I’ve seen this kind of thing up close: it’s a function of the drive and determination of the person involved, not a freak accident.

With that as prelude, here’s the next installment of the essay:

Develop a high-quality research environment

There is a considerable amount of research showing that people consistently underestimate the effect of the environment on personal effectiveness. This is particularly important in an academic environment where there are usually many short-term social pressures that are not directly related to research effectiveness – teaching, writing letters of recommendation and referee reports, committee work, academic politics. By contrast, in most institutions there are few short-term social pressures to do great research work.

Some of the highest-leverage work you can do involves improving your environment so that social pressures work for you as a researcher, rather than against you. Discussing this in detail would require another essay of length at least equal to that of the present one, but I will make a few remarks.

The first is that improving your environment is something anyone can do; students, in particular, often underestimate the magnitude of the changes they can bring about. Anyone can start a seminar series, develop a discussion area, create a lounge, organize a small workshop, or organize a reading group. Furthermore, although all these things are hard to do well, if you’re willing to do critical evaluations, experiment and try radical changes, preferably in partnership with equally committed people, things are likely to improve a great deal.

Second, institutions have long memories, so changes that you make in your environment will stick around for a long time. This means that once something is working well, chances are it’ll continue to work well without much help from you – and you can move on to improve some other aspect of your environment. Furthermore, each positive change you make actually improves your leverage with other people. I’ve known undergraduate students who had made so many creative positive contributions to their departments that their influence with canny senior faculty was comparable to the influence of other senior faculty.

You can now get your Erdos number down to two, which is rather better than the earlier auction run by William Tozier. (Via Ike Chuang).

Daniel Lemire, quoting Phil Agre, writes a nice post on academic freedom:

It seems you can either build an empire for the purpose of building an empire, because that’s your definition of success, or else, you can aim to remain “free”. That’s a very powerful idea:

You build networks around the issues you care about, you grow and change through the relationships that result, you articulate the themes that are emerging in the community’s work, and through community-building and leadership you get the resources to do the things that you most care about doing. It’s true that this method will never give you arbitrary power. But the desire for arbitrary power is not freedom – it is a particularly abject form of slavery. If you can let go of preconceived plans then you are free: you can choose whom to associate with, and as you build your network you multiply the further directions that you can choose to go. You also multiply the unexpected opportunities that open up, the places you can turn for assistance with your projects, the flows of useful information that keep you in contact with reality, the surveillance of the horizon that keeps you from getting cornered by unanticipated developments, and the public persona that ensures that people keep coming to you with offers that you can take or leave. That is what freedom is, and it is yours if you will do the work.

I give Agre a lot of credit from bringing in the concept of “freedom” in research. University professors will often talk about “academic freedom”. I think that freedom in research is a stronger form of freedom. You can have “academic freedom” but be a slave to the “publish-or-perish” paradigm for the power it brings you. Or else, you can “do the work”, that is, do your research as a network node, and leverage the strength of the network to make the research you want to do anyhow, much better, much stronger.

This, of course, is not the usual concept of academic freedom – the ability to speak truth to power, without losing your job. But it’s an important adjunct; if you need to give up your own freedom in an attempt to maintain power, then that is not freedom. It’s become corny, but I thought the old song had it about right – freedom can be just another word for nothin’ left to lose.

Via – d – I discover Edsger Dijkstra’s three golden rules for scientific research. It’s all interesting, but I particularly liked the first rule:

Raise your quality standards as high as you can live with, avoid wasting your time on routine roblems, and always try to work as closely as possible at the boundary of your abilities. Do this, because it is the only way of discovering how that boundary should be moved forward.

There’s a lot of truth in that.

The pedant in me can’t resist quibbling, at least a little: if you haven’t had a lot of success recently (or ever), spending some time on routine problems can help raise your morale, ensure you stay or come into contact with your community of researchers, and – most important – means that you earn your keep.

Developing research strengths

The foundation is a plan for the development of research strengths. What are you interested in? Given your interests, what are you going to try to learn? The plan needs to be driven by your research goals, but should balance short-term and long-term considerations. Some time should be spent on things that appear very likely to lead to short-term research payoff. Equally well, some time needs to be allocated to the development of strengths that may not have much immediate pay-off, but over the longer-term will have a considerable payoff.

In targeting areas of development, an important goal to keep in mind is that you want to develop unique combinations of abilities. You need to develop unique combinations of talents which give you a comparative advantage over other people. Do what you can do better than anybody; to mangle a quote from Lincoln, nobody can be better than everybody all of the time, but anybody can be better than everybody some of the time.

In my opinion the reason most people fail to do great research is that they are not willing to pay the price in self-development. Say some new field opens up that combines field X and field Y. Researchers from each of these fields flock to the new field. My experience is that virtually none of the researchers in either field will systematically learn the other field in any sort of depth. The few who do put in this effort often achieve spectacular results.

Finally, a note on how to go about developing some new research strength. A mistake I’m prone to make, and I know some others are as well, is to feel as though some degree of completeness is required in understanding a research field. In fact, in any given research field there are usually only a tiny number of papers that are really worth reading. You are almost certainly better off reading deeply in the ten most important papers of a research field than you are skimming the top five hundred.

These ideas carry over to the problem of staying current in your fields of interest: I believe that you can stay quite current by (a) quickly skimming a great deal of work, to keep track of what is known, and what sort of problems people are thinking about, and (b) based on that skimming, picking a dozen or so papers each year to read deeply, in the belief that they contain the most important research results of the year. This is not the only deep reading you’ll need to do; you’ll also need to do some which is related to the immediate problems that you’re working on. But you certainly should do some such deep reading.

>From the comments and Daniel Lemire’s blog:

When I started out doing research, I thought that research was about sitting in your office thinking up new ideas. God! Was I wrong!

Now, don’t get me wrong, research is not about having meetings with other researchers or spending time chatting, or drawing UML diagrams of what is to be done, or spending weeks on funding proposals. We might do these things, but they don’t make us good researchers. But neither will sitting in your office thinking new ideas. That’s not effective research.

On quasi-desert islands with no telecommunications, you’ll find very few great researchers. The social network doesn’t need to be immediate: I think you can be a great researcher even in a tiny school. And I don’t think your network should be made of students mostly, especially not your own students.

I believe the secret to being a good researcher is to belong to a tightly knitted group of solid researchers. Research is about networking. By tightly knitted, I don’t necessarily mean “military-like”: I mean that you feel peer pressure all the time to do good research. This can be achieved through emails, blogging, phone… whatever the mean…

Social forces are incredible things. As a researcher, being in a good social environment (which may need to be created from scratch) is like being a runner with a 50 kph tail wind. A turning point in the early days of my research group was the installation of a couple of sofas in front of a whiteboard in the room next to my office. It wasn’t my idea, but boy, was it about the best $30 we ever spent…

Self-development

Principles of personal change

If you always do what you’ve always done, you’ll always get
what you always got. – Australian netballer Vicki Wilson

How can we build personal habits that encourage research excellence? My belief is that the key is to examine in explicit detail our actualSet behavioural goals: To achieve meaningful personal change you need to know how you want to behave, what habits you want to have. Set goals for yourself. Write them down. Be precise. Some of the goals should be short-term; in fact, it’s best to start that way, since you can then get into the habit of improvement. Start small – there’s no need, initially, for a comprehensive program.

Set simple goals: If you set complex goals for yourself, they become difficult to evaluate, and difficult to think about in a day to day context (“Did I eat the right ratio of protein to carbohydrates to fat to salt today?”). Note, incidentally, that a simple change can still be a big change.

Make changes slowly: It’s better to make changes slowly, and do a good job, than to attempt grandiose changes which are so demanding that they can never be successfully implemented. As the old proverb goes, a journey of a thousand miles begins with a single step. As Benjamin Disraeli said, the secret of success is constancy of purpose; provided one holds constant to the purpose of becoming an effective researcher, small changes integrated over time will compound and result in tremendous improvement.

Evaluate the changes you make, and update your goals: To be effective, you have to evaluate the changes that you make. Say you set a goal to begin work by 6:30 am each day over the next week. This goal is of little use unless you keep a record of when you begin work each day, and then at the end of the week go through an evaluation process in which you first compare your goals to actual achievement, and then form an action plan, which may consist of either changing your goals, or of making a further change in behaviour in order to achieve the goal over the next week, or possibly doing both.

The process I’ve just described is, in my experience, a surefire way to personal change and growth. To conclude this section, I want to talk a little about some metaphors that I find useful when thinking about the process of personal change, and about some of the difficulties that crop up.

The first metaphor is that of the coach and a sporting team. As a researcher, one combines the roles of player and coach. A useful distinction to clarify thought is to divide your actions up into the roles of player and coach; think of having a player’s hat, and a coach’s hat. This metaphor sheds immediate light on one of the main difficulties faced in research, that of self-mastery and self-management. In a sporting team, there will always be difficult tasks and choices that, left to themselves, the players would be loath to take. Because there is an external force (the coach) imposing the actions, the difficult choices are often made anyway, to everyone’s long-run benefit. It’s easier to fool yourself and take the easy option than it is to fool anyone else; a good coach knows this, and works to prevent it. You need to become your own good coach.

The second metaphor is an idea from computer science known as the gradient descent algorithm. The gradient descent algorithm is a method for finding the maximum value of some function f(x) defined on a “landscape” of possible input values x. The way gradient descent works is to evaluate f(x) at some point, and then to make small perturbations x -> x’ in an attempt to find a value f(x’) larger than the initial f(x). The idea is that by following the local gradient we can find a maximum of the function. Self-improvement is similar, in that we make small changes in the way we work, evaluate whether this gives an improvement, and if so, continue moving in that direction. Indeed, this kind of change process can be applied in any area, not just personal development.

To finish off, I want to talk about one of the major pitfalls in achieving personal change, regression. Personally, I find it quite a downer when I begin developing some good new habits, things are going well, and then I find it all interrupted by some change in my routine. Maybe I go to a conference or on a holiday. My routine is disrupted, and when I return I find that the old good habits fall away. It’s tempting to get a bit down when this happens. While tempting, this, of course, is not a fruitful route to take.

I believe there are several ways one can combat this kind of regression. First of all, accept that this sort of regression will happen. We’re creatures of habit, and it’s easy to fall back into old habits, especially when those old habits require less immediate exertion on our own part. Second, it’s not a disaster when it happens. If you’ve learnt to do something once, you can learn to do it again; you just shouldn’t expect to be able to learn to do it overnight. It will take effort, just like it did the first time; a superficial effort is not enough, one must get back deeply into the process of change. The key is to ask oneself what good habits have been lost, and what sort of process can be used to get back into those habits. Then one has to pay the price again, in order to redevelop the habits. Personally, I am often far too impatient under these circumstances, and just expect to be able to go back to my old good habits without effort. The result, inevitably, is that I fail, and become unhappy about the failure, rather than paying the price necessary to get back into the good habits.

Note: This is part V of my continuing series. It, like the other parts, can for the most part be read independently. Part VI will arrive in the next day or two.

Aspects of research: self-development and the creative process

Research involves two main aspects, self-development and the creative process of research. We’ll discuss the specifics of each aspect below, but for now I want to concentrate on the problem of achieving balance between the two, for I believe it is a common and significant mistake to concentrate too much on one aspect to the exclusion of the other.

People who concentrate mostly on self-development usually make early exits from their research careers. They may be brilliant and knowledgeable, but they fail to realize their responsibility to make a contribution to the wider community. The academic system usually ensures that this failure is recognized, and they consequently have great difficulty getting jobs. Although this is an important problem, in this essay I will focus mostly on the converse problem, the problem of focusing too much on creative research, to the exclusion of self-development.

There are a lot of incentives for people to concentrate on creative research to the exclusion of self-development. Throughout one’s research career, but particularly early on, there are many advantages to publishing lots of papers. Within limits, this is a good thing, especially for young researchers: it brings you into the community of researchers; it gives you the opportunity to learn how to write well, and give good presentations; it can help keep you motivated. I believe all researchers should publish at least a few papers each year, essentially as an obligation to the research and wider community; they should make some contribution, even if only a small one, on a relatively unimportant topic.

However, some people end up obsessed with writing as many papers as possible, as quickly as possible. While the short-term rewards of this are attractive (jobs, grants, reputation and prizes), the long-term costs are significant. In particular, it can lead to stagnation, and plateauing as a researcher. To achieve one’s full potential requires a balancing act: making a significant and regular enough research contribution to enable oneself to get and keep good jobs, while continuing to develop one’s talents, constantly renewing and replenishing oneself. In particular, once one has achieved a certain amount of job security (a long-term or permanent job) it may make sense to shift the balance so that self-development takes on a larger role.

For many people (myself included) who have concentrated mainly on making creative research contributions earlier in their careers, this can be a difficult adjustment to make, as it requires changing one’s sense of what is important. Furthermore, there is a constant pull towards concentrating on research over self-development, since there are often short-term incentives to sacrifice self-development for research (“I’ve got to get this paper out now”), but rarely vice versa. To balance these tendencies, we need to remember that nobody, no matter how talented, is born an effective researcher; that distinction can only be obtained after a considerable amount of hard work and personal change, and there is no reason to suppose that just because one is now able to publish lots of papers that one has peaked as a researcher.

In my opinion, creative research is best viewed as an extension of self-development, especially an extension of a well-developed reading program. I don’t believe the two can be completely pried apart, as the two interact in interesting non-linear ways. I’m now going to talk in a little more detail about both processes, keeping in mind that the ultimate goal of research is new ideas, insights, tools and technologies, and this goal must inform the process of self-development.

I once heard Hofstadter’s Law of computer software engineering:

It will always take longer than you think, even when Hofstadter’s Law is taken into account.

Having just spent the day thinking and talking about fault-tolerant quantum computing, I think there’s a corresponding Hofstadter’s Law of quantum computer hardware engineering:

It will always be more difficult than you think, even when Hofstadter’s Law is taken into account.

An introductory lecture on fault-tolerant quantum computing, for a local mini-workshop on fault-tolerance that we’re holding at UQ today. This gives a very high level picture of fault-tolerance and the threshold theorem; with a few modifications, you could give this lecture to a group of bright first-year undergrads. Even after nearly 10 years, I find it an incredible result: you can build a reliable computer (quantum or classical) from faulty components.

Note that for some reason Acrobat choked on the file, so it’s just in Powerpoint, not pdf.

My essay on research effectiveness will continue tomorrow.