What the number of citations can be made to suggest by tweaking them

The number is really like a tongue-in-cheek tribute to the man, not a serious measure of achievement. And yet, it touches on a serious question: is there a way to measure how good, or effective, a scientist is? A low Erdös number means you have worked with some serious mathematicians, so that does indicate that you have some worth as a mathematician yourself. But still, it really is just in the nature of a tribute to a man who touched so many.

There is, though, a more serious measure that’s often used: How many times a paper you have authored has been cited in other papers by other scientists. You can probably tell that this “citation index” carries some weight. For if another scientist cites something you have researched and written up, it means that scientist found your work relevant and useful in his work. And if several scientists cite your paper, it means you produced something of some relatively wide relevance. If your paper continues to be cited long after it is published, maybe even after you’re dead and gone, that speaks of the lasting impact of your findings.

Of course, this idea of an index can be tweaked. For example, what’s the calibre of the citations of your paper? Should a reference in Mint, for example, count the same as a reference in Nature, or Scientific American? Does a Mint reference mean that a wider audience than just academia is reading your paper? If so, surely that suggests a broader understanding and appeal? Besides, how good are the references you yourself cite—meaning, how many of the important results in your field are you aware of while you do your research?

Considerations like these go into the calculation of various metrics—called h-index, g-index and more. The h-index, for example, is calculated thus: Of all a scientist’s published papers, if some number “h” of them each have h or more citations, and the rest have h or less citations, then her h-index is h. So let’s say researcher Sharvari has published 30 papers. Seven of them have each been cited at least seven times each; the other 23 are each cited seven or fewer times. Sharvari then has a h-index of 7.

To give you a quick idea, Erdös has a h-index of 76, Albert Einstein 92. Einstein’s E = mc2 paper has been cited nearly 500 times; his special relativity paper over 2,000 times.Like all statistical measures, these are used and interpreted in different ways. Certainly, Einstein should figure at or near the top of any ranking of scientists. But considering the sheer number of scientific journals that there are, some no doubt of dubious merit, it should hardly be a surprise to find unknown or unexpected names on such rankings. These might be solid scientists who are just not widely known; they might also be not-so-solid ones who find ways to inflate their citation indices.

And in fact, one recent paper addresses exactly the use and misuse of such citation metrics, apparently seeking a way to rationalize them (September 2022 data-update for Updated science-wide author databases of standardized citation indicators, John P.A. Ionaddis, Elsevier, 10 October 2022, https://bit.ly/3UliNVw). It looks at citation data for about 200,000 scientists around the world and attempts to rank them according to a composite—called the c-score—of various citation indices like the h-index. As the paper explains, “The c-score focuses on impact (citations) rather than productivity (number of publications).”

It’s not clear to me how many scientists, whether on this list or not, pay attention to a list like this. I suspect the more serious ones simply go about their work, uninterested in such rankings. But be that as it may, how did Ionaddis choose those approximately 200,000 scientists?

Let me quote his paper: “The selection is based on the top 100,000 scientists by c-score … or a percentile rank of 2% or above … 200,409 scientists are included in the single recent year dataset.”

Never mind “single recent year”. The language seems calculated to obfuscate this process: In particular, how does 100,000 become 200,409?

Perhaps it’s the percentile rank of 2%? That may be, because that number takes in those who are above the bottom 2% in the rankings (what “2 percentile” means); in other words, the 200,000 make up the top 98% of scientists in the overall list. To me, this seems the most reasonable interpretation of this paper.

This is all leading somewhere, I promise. I was prompted to do this digging by a recent large newspaper ad for Patanjali products featuring this claim: “Pujya Acharya Balkrishna ji and the Patanjali (PRI) among top 2% scientists in the world.” Patanjali’s website sent me to the Ionaddis paper above, and from there to two enormous Excel files which contain data about all these scientists.

The paper’s only mention of 2% is “percentile rank of 2% or above”. So “top 2% scientists” in the ad should actually be “top 98% scientists”. Or, equivalently, “scientists ranked higher than the bottom 2%.”

Neither of which, you’ll agree, is quite as eye-catching as “top 2%”.

But wait, where exactly in the rankings is Acharya Balkrishna? With a c-score “rank” of 367,268, he appears in the 192,004th row, of a total of 200,409.

That is, 95.8% of these scientists are ranked higher than Acharya Balkrishna. That is, his percentile rank is 4.2. That is, only 4.2% of the scientists rank below him. Give that a thought.

Once a computer scientist, Dilip D’Souza now lives in Mumbai and writes for his dinners. His Twitter handle is @DeathEndsFun.