Higher Education Strategy Associates

More Shanghai Needed

I’m in Shanghai this week, a guest of the Center for World-Class Universities at Shanghai Jiao Tong University for their biannual conference. It’s probably the best spot on the international conference circuit to watch how governments and institutions are adapting to a world in which their performance is being measured, compared and ranked on a global scale.

In discussions like this the subject of rankings is never far away, all the more so at this meeting because its convenor, Professor Nian Cai Liu, is also the originator of the Academic Ranking of World Universities, also known as the Shanghai Rankings. This is one of three main competing world rankings in education, the others being the Times Higher Education Supplement (THES) and the QS World Rankings.

The THES and QS rankings are both commercially-driven exercises. QS actually used to do rankings for THES, but the two parted ways a couple of years ago when QS’s commercialism was seen to have gotten a little out of hand. After the split, THES got a little ostentatious about wanting to come up with a “new way” of doing rankings, but in reality, the two aren’t that different: they both rely to a considerable degree on institutions submitting unverified data and on surveys of “expert” opinion. Shanghai, on the other hand, eschews surveys and unverified data, and instead relies entirely on third-party data (mostly bibliometrics).

In terms of reliability, there’s really no comparison. If you look at the correlation between the indicators used in each of the rankings, THES and QS are very weak (meaning that the final results are highly sensitive to the weightings), while the Shanghai rankings are very strong (meaning their results are more robust). What that means is that, while the Shanghai rankings are an excellent rule-of-thumb indicator of concentrations of scientific talent around the world, the QS and THES rankings in many respects are simply measuring reputation.

(I could be a bit harsher here, but since QS are known to threaten academic commentators with lawsuits, I’ll be circumspect.)

Oddly, QS and THES get a lot more attention in the Canadian press than do the Shanghai rankings. I’m not sure whether this is because of a lingering anglophilia or because we do slightly better in those rankings (McGill, improbably, ranks in the THES’s top 20). Either way, it’s a shame, because the Shanghai rankings are a much better gauge of comparative research output, and with its more catholic inclusion policy (500 institutions ranked compared to the THES’s 200), it allows more institutions to compare themselves to the best in the world – at least as far as research is concerned.

This entry was posted in Rankings and tagged , , , . Bookmark the permalink.

One Response to More Shanghai Needed

  1. Phil Baty says:

    A nice, fun piece Alex.

    I welcome engaged criticism of the rankings, that’s how we ensure that we continue to improve, but I think your case against the Times Higher Education World University Rankings would be better made if:

    a) You cited our results correctly. You say that McGill is “improbably” ranked in our top 20. This would indeed be improbable, because McGill is actually ranked 28th in the world by THE in 2011-12. See: http://bit.ly/thewur
    b) You were more aware of our offering: this year we published a list of 400 institutions (not 200 as you state). This is not far from Shanghai’s 500, and we actually give firm ranking positions to the top 200, giving more detailed data on more institutions than Shanghai.
    c) You got our name right. The magazine which publishes the rankings is known as “Times Higher Education”. We were once called “THES”, but we have been known as Times Higher Education – “THE” – since 2008.

    It is correct that THE decided to end its partnership with QS in 2009, in order to produce a more robust piece of work with a new partner, Thomson Reuters. This is something we have achieved and we are delighted with the results of the new THE-Thomson Reuters partnership.

    Shanghai’s rankings are admirable and robust – as long as you only want a very, very narrow sense of research power in the sciences. Only THE produces a ranking that looks properly at all core missions of a global university – research, teaching, knowledge transfer and internationalization. Have a closer look at what we do here: http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=415715&c=1

    I’d be very happy to offer you and your clients a free webinar so they can better understand what we do.

Leave a Reply

Your email address will not be published. Required fields are marked *

We encourage constructive debate. Therefore, all comments are moderated. While anonymous or pseudonymous comments are permitted, those that are inflammatory or disrespectful may be blocked.