{"id":248,"date":"2008-01-31T10:30:10","date_gmt":"2008-01-31T14:30:10","guid":{"rendered":"http:\/\/michaelnielsen.org\/blog\/?p=248"},"modified":"2008-01-31T11:59:01","modified_gmt":"2008-01-31T15:59:01","slug":"why-the-h-index-is-virtually-no-use","status":"publish","type":"post","link":"https:\/\/michaelnielsen.org\/blog\/why-the-h-index-is-virtually-no-use\/","title":{"rendered":"Why the h-index is little use"},"content":{"rendered":"

In 2005 Jorge E. Hirsch published an article in the Proceedings of the National Academy of Science (link<\/a>), proposing the \u00e2\u20ac\u0153h-index\u00e2\u20ac\u009d, a metric for the impact of an academic\u00e2\u20ac\u2122s publications.<\/p>\n

Your h-index is the largest number n<\/em> such that you have n<\/em> papers with n<\/em> or more citations. So, for example, if you have 21 papers with 21 or more citations, but don’t<\/em> yet have 22 papers with 22 or more citations, then your h-index is 21.<\/p>\n

Hirsch claims that this measure is a better (or at least different) measure of impact than standard measures such as the total number of citations. He gives a number of apparently persuasive reasons why this might be the case.<\/p>\n

In my opinion, for nearly all practical purposes, this claim is incorrect<\/strong>. In particular, and as I’ll explain below, you can to a good approximation work out the h-index as a simple function of the total number of citations, and so the h-index contains very little information beyond this already standard citation statistic.<\/p>\n

Why am I mentioning this? Well, to my surprise the h-index is being taken very seriously by many people. A Google search<\/a> shows the h-index is spreading and becoming very influential, very quickly. Standard citation services like the Web of Science now let you compute h-indices automatically. Promotion and grant evaluation committees are making use of them. And, of course, the h-index has been extensively discussed in the blogosphere (e.g., here<\/a>, here<\/a>, here<\/a>, here<\/a>, here<\/a>, here<\/a>, and here<\/a>). <\/p>\n

Hirsch focuses his discussion on physicists, and I’ll limit myself to that group, too; I expect the main conclusions to hold for other groups, with some minor changes in the constants. For the great majority of physicists (I\u00e2\u20ac\u2122ll get to the exceptions), the h-index can be computed to a good approximation from the total number of citations they have received, as follows. Suppose T<\/em> is the total number of citations. For most physicists, the following relationship holds to a very good approximation:<\/p>\n

(*) h ~ sqrt(T)\/2<\/em>.<\/p>\n

Thus, if someone has 400 citations, then their h-index is likely to be about half the square root of 400, which is 10. If someone has 2500 citations, then their h-index is likely to be about half the square root of 2500, which is 25.<\/p>\n

The relationship (*) actually follows from the data Hirsch analysed. He notes in passing that he found empirically that T = a h^2<\/em>, where a<\/em> is between between 3 and 5. Inverting the relationship, we find that (*) holds to within an accuracy of about plus or minus 15%. That\u00e2\u20ac\u2122s accurate enough \u00e2\u20ac\u201c nobody cares whether your h-index is 20 or 23, particularly since citation statistics are already quite noisy. Provided a<\/em> is in this range, h<\/em> contains little additional information beyond T<\/em>, which is already a commonly used citation statistic.<\/p>\n

What about the exceptions to this rule? I believe there are two main sources of exception.<\/p>\n

The first class of exception is people with very few papers. Someone with 1-4 papers can easily evade the rule, simply because their distribution of citations across papers may be very unusual. In practice, though, this doesn\u00e2\u20ac\u2122t much matter, since in such cases it\u00e2\u20ac\u2122s possible to look at a person\u00e2\u20ac\u2122s entire record, and measures of aggregate performance are not used so much in these cases, anyway.<\/p>\n

The second class of exceptions is people who have one work which is vastly more cited than any other work. In that case the formula (*) tends to overstate the h-index. The effect is much smaller than you might think, though, since it seems to be that for the great majority of physicists their top-cited publication has many more citations than their next-most cited publication.<\/p>\n

In any case, I hypothesize that this effect is mostly corrected by using the formula:<\/p>\n

(**) h approx b sqrt(T\u00e2\u20ac\u2122)<\/em><\/p>\n

where T\u00e2\u20ac\u2122<\/em> is a the total number of citations, less the most cited publication, and b<\/em> is a constant which needs to be empirically determined. At a guess I\u00e2\u20ac\u2122d believe that omitting the top two cited publications would work even better, but after that we\u00e2\u20ac\u2122d hit the point of diminishing returns.<\/p>\n

Returning to the main point, my counter-claim to Hirsch is that the h-index contains very little additional information beyond the total number of citations. It\u00e2\u20ac\u2122s not that the h-index is irrelevant, it\u00e2\u20ac\u2122s just that in the great majority of cases the h-index is not very useful<\/em>, given that the total number of citations is likely to already be known.<\/p>\n","protected":false},"excerpt":{"rendered":"

In 2005 Jorge E. Hirsch published an article in the Proceedings of the National Academy of Science (link), proposing the \u00e2\u20ac\u0153h-index\u00e2\u20ac\u009d, a metric for the impact of an academic\u00e2\u20ac\u2122s publications. Your h-index is the largest number n such that you have n papers with n or more citations. So, for example, if you have 21… Continue reading Why the h-index is little use<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-248","post","type-post","status-publish","format-standard","hentry","category-3","entry"],"_links":{"self":[{"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/posts\/248","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/comments?post=248"}],"version-history":[{"count":0,"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/posts\/248\/revisions"}],"wp:attachment":[{"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/media?parent=248"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/categories?post=248"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/michaelnielsen.org\/blog\/wp-json\/wp\/v2\/tags?post=248"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}