Measuring Efficiency: A Comparison of Multilevel Modelling and Data Envelopment Analysis in the Context of Higher Education

Data envelopment analysis (DEA) and multilevel modelling (MLM) are applied to a data set of 54,564 graduates from UK universities in 1993 to assess whether the choice of technique affects the measurement of universities’ performance. A methodology developed by Thanassoulis and Portela (2002; Education Economics, 10(2), pp. 183–207) allows each individual's DEA efficiency score to be decomposed into two components: one attributable to the university at which the student studied and the other attributable to the individual student. From the former component, a measure of each institution's teaching efficiency is derived and compared to the university effects from various multilevel models. The comparisons are made within four broad subjects: pure science, applied science, social science and arts. The results show that the rankings of universities derived from the DEA efficiencies which measure the universities’ own performance (i.e., having excluded the efforts of the individuals) are not strongly correlated with the university rankings derived from the university effects of the multilevel models. The data were also used to perform a university-level DEA. The university efficiency scores derived from these DEAs are largely unrelated to the scores from the individual-level DEAs, confirming a result from a smaller data set (Johnes, 2006a; European Journal of Operational Research, forthcoming). However, the university-level DEAs provide efficiency scores which are generally strongly related to the university effects of the multilevel models.