Home / Addressing the Academy / IMPACT! Academics, Citation, and Scholarly Self Delusion

IMPACT! Academics, Citation, and Scholarly Self Delusion

Greetings boys and girls, in today’s lesson I want to talk about “citation impact factor.” And just what is citation impact factor you ask?  Well, impact factor is what scientists use to assess their “impact” on things. It is a measure, I suppose, of a scientist’s “value” as scientist. Despite some recent attempts to kick it up a notch, citation impact is a very simple measure. You write an article, you publish it, and you count how many other scientists “cite” your article (i.e. include it in the reference section of their article or book). That’s all there is to it. So for example if I write an article on labour process theory, and if I publish it in the journal, Work, Employment, and Society (Sosteric, 1996), and if that article goes on (for a time) to become one of the top ten most cited articles in the journal (as it did), perhaps gathering as many as (gasp) 99 citations, then it is judged to be a high impact article.

Yay!

If its high impact I get to point to it and say “I wrote something important.” If its not high impact then all I can do is hang my head in shameful scholarly obscurity, presumably confined to the dustbins of this planet’s scientific history.

Yawn.

As boring as it is, it seems sensible and logical enough, if you don’t know any better, but I think there are problems. For example, if you unpack the notion of “impact” a bit and ask “impact on what,” the validity of the whole enterprise comes into question. I mean, what can you really say about an article that has been cited 99 times? Does that mean it has been read 99 times? Hardly! Many scholars cite articles based on only a quick read of the abstract, and many more read things that they never cite at all. I mean, I’ve read thousands of books and articles in my life time and I’ve cited only a handful. Try as we might, it hardly seems right, or fair, to suggest that “number of citations” means anything at all in those circumstances. And these are just the most obvious problems. Even if you get past this rather glaring “fail” to the point where you would suggest that having a 99 citation count actually means something , you still have to answer the questions “means what?” And that’s not so easy to answer either. Does having a 99 mean your ideas are part of the intellectual fabric of this planet? Maybe! But then again, maybe not. You can’t tell that from the number alone. In order to determine actual impact you would have to do some complicated (and expensive) qualitative research, and if you did I’m pretty sure you’d find that sometimes an idea is incorporated into the fabric after 99 citations, and sometimes not. In other words if you did the research you’d find the citation measure is unreliable, and invalid. And as any methodologist will tell you, if a measure is unreliable and invalid, it’s totally and utterly useless.

Even what I’ve said so far is enough to call into question the validity of citation counting, but some people might want to grasp and claw at the measure anyway. There is, however, a final consideration that makes it clear just how ridiculous citation counting is and that is the fact that sometimes scientific ideas can have major impact without ever getting cited at all. Consider this. I was heavily influenced by the thinking of Karl Marx, but I don’t cite him in every article I write. I don’t feel I have to, but his influence is there all the time. And if that doesn’t cause you to call into question the citation analyst’s little radar, consider this. A few years ago I wrote an article on forms of emotional abuse commonly found in schools (Sosteric, 2012). In the time that it has been on this site many people have commented on it, and many hundreds of thousands of people (parents and teachers) have read it. Just last year it was picked up by Educational Testing Service in the U.S.A to be used in their teacher certification materials. The contract I signed specified 50,000 teachers over the next ten years! This means that my ideas on emotional abuse in schools will be influencing the next generation of teachers as they complete their programs and pass certification. Clearly this article on abuse in schools is having a major impact, but AFAIK it hasn’t been cited once.

So where does all this leave citation impact factor? If you ask me, it is an unreliable, invalid, ridiculous, and useless measure.  And frankly, I’m not the only one to say so (Adam, 2002; Baum, 2013; “Not-so-deep impact,” 2005; “The ratings game,” 2010). When you start to get really deep, the issues keep piling higher. Citation counts may give the university administrator something to point to when they’re looking at you for a promotion (which to be honest is what they are really all about), but other than that they have no utility at all. You won’t get a fair representation of your impact by a citation count, so don’t even bother to try. You also shouldn’t use it to prove just how important you are, because its a vacuous and meaningless measure that speaks more to academic ego than it does to actual contribution. If you want my advice, just give it up.

References

Adam, D. (2002). The counting house. Nature, 415(6873), 726-729. doi: http://dx.doi.org/10.1038/415726a

Baum, J. A. C. (2013). The excess-tail ratio: correcting journal impact factors for citation distributions. [Article]. M@n@gement, 16(5), 697-706.

Not-so-deep impact. (2005). [10.1038/4351003b]. Nature, 435(7045), 1003-1004.

The ratings game. (2010). [10.1038/464007b]. Nature, 464(7285), 7-8.

Sosteric, M. (1996). Subjectivity and the Labour Process: A Case Study in the Food and Beverage Industry. Work, Employment, and Society, 10(2).

Sosteric, M. (2012). The emotional abuse of our children: Teachers, schools, and the sanctioned violence of our modern institutions. The Socjournal, March.

 

 

Cite This Article

Dr. S. (2014). IMPACT! Academics, Citation, and Scholarly Self Delusion. The Socjourn. [https://sociology.org/citation-impact-factor-a-scholars-cautionary-tale/]

Lila, the Revolutionary

By: William T. Hathaway

Lila, the Revolutionary is a fable for adults about an eight-year-old girl—smart, charming, and tough as can be—who creates a world revolution for social justice. No one ever told her she couldn't end poverty and inequality, so she doesn't doubt that she can Just Do It! Starting with the Nike shoe factory where she works. Like the boy in "The Emperor's New Clothes," Lila can see the reality that adults are blind to. And she's not shy about pointing it out. Her story is a call to action: If Lila can do it, so can we. She convinces us that Yes, a better world is possible, and we're the ones to create it.


[ Kindle | Amazon.com | Book Finder ]

About Dr. Michael S. (Dr. S)

Michael S. (Dr. S.) is a scientist, sociologist, author, mystic, and mystical poet whose interests are human psychology, human society, spirituality, consciousness, global pedagogy, and global transformation. He's busy writing about a dozen books all of which are aimed at enlightening the people and transforming the planet in line with the purpose, and for the benefit of, All. For more from Michael, visit michaelsharp.org, sociology.org, or his personal favorite, The Lightning Path.

3 comments

  1. I thought citing was more about proving the legitimacy of your statements in a writing. This, of course, never made sense for someone who often writes about issues where the legitimacy lies within personal experience. (I have actually cited some of your writing BTW). The impact of citing must be for more about engagement at social luncheons served with finger sandwiches. This is probably why I would never attend one and will more than likely leave this life without making an impact.

    • Well it is. When I cite someone else it is because I’m drawing their argument or thinking in, acknowledging their contribution, or building an argument in short hand. I don’t have any problem with the practice of citation, what I don’t like is when people count how many citations an article has as some kind of “proof” it means something. My point here is that citation counting, at best, doesn’t mean anything. At worst it is a powerful method of social control.

      Nice to see you still around!

      • Nice to see you still around, too! I enjoy your articles. It is discouraging to know that good, informative writing such as yours, has been diminished to a popularity contest with intentions to keep the masses unconscious. Although I am unfamiliar with the process of citation counting as legitimacy of scholarly impact, it does sound much like it amounts nothing more than the social media tool of ‘liking something’.

        Well, in any case, you get my thumbs up!

Leave a Reply to lucy Cancel reply

Your email address will not be published. Required fields are marked *

*