When is a cow not spherical: public discourse and the dangers of implicit models

There�s been a lot of fuss within the blogosphere and the US media about comments made by Harvard President Larry Summers, about women and academia.

I�m not going to comment directly on that here, as all that I have to say has been said much better elsewhere. In particular, Sean Caroll has an excellent post (scroll down to the post “Sex and Science!”) on the subject.

What I want to talk about in this post is a pitfall in the way much of the broader online discussion has been framed.

In describing this pitfall I am to a large extent preaching to the choir� I expect most of my readers understand perfectly well what I�m about to say. I�m writing it, then, simply because I think it is relevant both to the Summers incident, and to many other public discussions, and because it�s evident that too many of the people involved in public debate either aren�t aware of this problem, or have simply forgotten it.

(A secondary reason, as with much of my writing, is to clarify the thoughts in my own head!)

A lot of the discussion about the Summers incident has focused on the question �Are women intrinsically better / worse than men in academic job roles?�, and variants thereof.

Now what exactly does that question mean?

In fact, it can mean many different things to different people. My guess is that when most people say that, they�re thinking (at least vaguely) in terms of some simple underlying model.

Let me give an example of what I mean by an �underlying model�. It�s a mathematical model, but it�s a very simple one, so even if you�re not mathematically inclined, please bear with me: the point will (I hope) be clear anyways.

In this model each person has an �intrinsic ability for academia�, a point score (between 1 and 100, say).

The Male population has some distribution (say, Gaussian) with a mean M and a standard deviation S.

The Female population has some distribution (again, Gaussian) with a mean F and the same standard deviation S.

What I think many people implicitly have in mind when they use phrases like �men are better suited than women for academic jobs� is that M > F, i.e., that men are on average better than women.

When phrased in these quantitative terms, though, you quickly see a problem.

The problem arises when you make the model just a tiny bit more complicated � and probably more realistic.

Let�s suppose in the revised model we still have Gaussian distributions, with the male mean M = 50 and the female mean F = 48, so men are �better� (on average) than women.

(Disclaimer: I�ve chosen those numbers ad hoc as representative of the hypothesis many people have put forward online, not because I believe that men are on average better than women.)

But suppose the standard deviations for the two populations are not the same. Suppose instead that the male standard deviation is 10 points, and the female standard deviation is 12.

If the median Harvard Faculty member needs a score of 80 points, then there will be many more women with the requisite ability than men, even though, on average, men have a higher points score.

It gets more complicated.

Suppose instead that you consider a local community college, where the median Faculty score is (say) 58. In that case, more men than women will have the ability to be on the Faculty at the community college, but more women than men will have the ability to be on the Faculty at Harvard.

All of a sudden whether you say �men are more suited than women to academia� depends an awful lot on how you define �academia�. More importantly, the conclusions you draw about policy may depend on other value judgements that you might have thought were unrelated to the original question: like whether it�s more important to foster community colleges or elite research Universities like Harvard.

In fact, it�s even worse than that. Both my models are ridiculously oversimplified � even though I�ve argued that my second model is probably a good bit better than what some people are using. You can�t quantify this kind of thing with a single score, we don�t have Gaussian populations, and so on. Real life will be much more complicated than even my second model suggests.

The problem I�m pointing out is several-fold, and it applied to lots of public discussion, not just of intrinsic differences between men and women, but of other topics.

First, underlying many such discussions are implicit models which are rarely if ever articulated. Indeed, people often don�t realize that they�re working off such implicit models � usually vaguely defined, vastly oversimplified models � rather than reality. This can lead to all sorts of mistakes and omissions.

Second, there tends to be a lot of variation in what models different people believe. This can lead even parties of good faith to have trouble understanding one another (much less coming to agreement), especially if they don�t understand that such differences are possible. People (and I’m certainly guilty)have a disturbing tendency to confuse their models with reality.

What�s the point of all this? It isn�t that every time some question like this comes up, we should all spend our time writing down mathematical models, comparing their various merits, and so on, or that the key is to have the “right model” or anything like that.

The point is simply to remember that public discussion is carried on using implicit models of this type. Unless we are aware of the existence of those models, and the difficulties they can cause, both due to miscommunication and due to mistakes arising from confusing models with reality, having a productive substantive discussion about any contentious issue is extremely difficult.

One final comment. It’s hard not to give the impression in this post that �of course, I�d never do such a silly thing�. Of course, I would, although I hope I�d catch myself at least some of the time. However, if you want an amusing (and sobering) example along similar lines, involving a physicist and an economist, see the first few pages of Dietrich Dorner�s excellent book “The Logic of Failure”. You can actually read the passage in question by using amazon.com�s �Search Inside this Book� feature, and searching for �physicist�; you�ll find an entry point to the Front matter, which is where you want to be. More generally, Dorner�s book makes a pretty compelling case that we all make mistakes like this all the time, even in very simple cases.