In ‘A New Approach to University Rankings’ published in the January edition of  Springer's Higher Education,  Dr Christopher Tofallis (right) of the Deptment of Statistics, Economics, Accounting and Management Systems. shows an alternative way to calculate league table results, avoiding the problem of having to add ‘apples and pears’ – quantities measured in different units.

By using data published in The Complete University Guide, Dr Tofallis shows a comparison to current league table calculations against his own methodology, which raises the question – are current university rankings a fair representation of UK institutions?

Dr. Tofallis says: “The paper focuses on the issue of how different measures are aggregated.  All current publishers of league tables use an additive approach which includes a normalisation step to allow the data to become ‘comparable’. The problem is that there are different ways of achieving this comparability, and each way leads to a different ranking.”

The paper proposes a multiplicative approach to aggregation, which overcomes these difficulties. The attraction of multiplying the data, rather than adding it together, means a normalisation step is not required. The idea for this method of combining variables measured in different units comes from the way it is achieved in scientific equations.

 “When the multiplicative approach is used the fact that some variables are numerically much greater than others does not matter since a rescaling of any variable (by multiplying by a positive constant) would have no effect on the rankings. For example, consider a switch from measuring expenditures in thousands of pounds to pounds; this would simply lead to a multiplication of the score by 1000 for every institution.” he notes.

Dr Tofallis’ methodology is very general and can be applied to many other types of ranking problem, forming part of wider research that will see a similar formula applied to other large data sets.

3D RANKING AND RESPECTFUL 

Endorsing Dr Toffalis arguement and methodology is the United Nations that publishes the Human Development Index, a ranking of countries based on an aggregate of three dimensions using an arithmetic mean, i.e. additive aggregation.

At the end of 2010, after 20 years  of using this approach the United Nations decided that it would be an improvement to switch to a geometric mean, i.e. multiplicative aggregation

"We reconsidered how to aggregate the three dimensions. A key change was to shift to a geometric mean, thus in 2010 the HDI is the geometric mean of the three dimension indices.

Poor performance in any dimension
is now directly reflected in  HDI, there is no longer perfect substitutability. This method captures how well rounded a country is  across the three dimensions.
Above: Courtesy Worldshap.in

As a basis for comparisons of achievement, this method is also more respectful of the intrinsic differences in the dimensions than a simple average is. It recognises that health, education and income are all important, but also that it is hard to compare these different dimensions of well-being.”
(United Nations Development Programme, 2010, p.15)