Ranking the World's Universities

Leaders and administrators from hundreds of universities worldwide convened in Kuala Lampur to debate world university rankings and discuss the methodology used to determine these rankings.  Details of the event, including powerpoint slides and pictures, can be found here.

Although university rankings are always controversial, world university rankings have become even moreso because of the current efforts of major study destinations like the US, UK, and Canada to promote their higher education systems to foreign students.  Rankings have become yet another arena in the ever-expanding competition for foreign students, with each country proudly announcing the number of universities it has placed in the top 100.

However, not all rankings are created equal, as demonstrated by the sometimes divergent results yielded by the two most esteemed ranking systems: the Academic Ranking of World Universities (ARWU) and the THE – QS World University Rankings.  For example, the ARWU has Stanford University at #2, whereas the THE-QS has Stanford at #16.  Perhaps Stanford alums Larry Page and Sergei Brin of Google teach the THE-QS a little about organizing and ranking information?

This divergence might be explained by the unique methodology of each system.  The ARWU focuses on academic accolades and notoriety, giving 40% weight to the quality of faculty (determined by number of faculty winning nobel prizes/fields medals and number of highly-cited faculty) and 40% to research output (determined by number of faculty papers published in prestigious academic journals).

The THE-QS rankings focuses on academic peer assessment (40%), student/faculty ratio (20%), and research output (20%).  Although the THE-QS is more broad in its focus, it has been criticized for focusing too much on academic peer assessment, which is very subjective.

The QS survey selects 1800 respondents from around the world and asks them to fill out a survey evaluating the strongest schools in each fields in terms of research and relative student quality.  It first asks them to indicate the subject areas with which they are most familar, never requiring any verification of their familiarity.  It then asks them to pick the 30 strongest schools in each of the subject areas they indicated familiarity with.  The surveys are then adjusted to prevent geographical bias.  The more a school was picked in a particular field, the higher the school’s academic peer review score in that field.

Since the the northeastern US and England contain the largest concentration of universities in the world, many of the 1800 respondents in the directory came from these universities.  Not suprisingly, the peer review results favored older universities clustered in the Northeastern United States and England.  This likely explains Stanford’s relegation to #16, as it is located in Palo Alto, California, far from the old college towns of the northeast.  

However, the ARWU is not without its faults.  First, it makes no attempt to measure the quality of the teaching at the universites.  Also, the weight given to the number of nobel prize winning faculty seems excessive.  Though the conception might seem a little narrow, the execution is excellent.  It is undeniably objective and transparent.

Both rankings focus on reputation but the ARWU does a better job because it focuses measures the actions of experts while they are in their expert capacity.  This is because the ARWU employs pre-existing assessments of quality to determine reputation: the decision of a journal editor to publish an article or the decision of a researcher to cite a professor’s work.  These decisions involve the professional fates of the respondents.  The THE – QS rankings fall short in this regard because the respondents have no incentive to answer truthfully, or more importanly, to admit that they don’t know.

Does this even matter, you ask?  Isn’t perception all that matters anyways?  Perception, counts for a lot in the fields of law, finance, and politics.  But the benefits of perception extends only so far, just around the point where the network ends.

Ironically, it is the metric most important for many undergraduates, reputation among employers, that is given the least weight (10% by the THE-QS and 0% by the ARWU). 

What does this mean for the average student?  To be wary of rankings and know precisely why you are using the rankings.  For now, anybody planning to major in the math and sciences should pay attention the ARWU rankings, as they will only become more influential in separating math and science programs.  If you plan on doing law or finance in one of the Alpha World Cities, the THE – QS top 20 will do you just fine, otherwise don’t bother.  But if you really want to hedge your bets, just go to Harvard.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: