Study finds link between uni rankings and QS contracts
Published 27/04/2021
Research from UC Berkeley’s Center for Studies in Higher Education has claimed that Russian universities that have contracts with rankings creator QS do significantly better in rising up the QS World Rankings than universities that do not work with the group.
“Results show that universities with frequent QS-related contracts experienced much greater upward mobility in both overall rankings and in faculty-student ratio scores over five years in the QS World Rankings,” noted CSHE in an email.
It added that the report used data on the positions of 28 Russian universities in QS World University Rankings from 2016 to 2021, their contracts for services from QS and then compared it with data from Times Higher Education rankings and national statistics.
“Results show that universities with frequent QS-related contracts experienced much greater upward mobility in both overall rankings”
“Positions of Russian universities that had frequent QS-related contracts increased on 0.75 standard deviations (approximately 140 positions) more than they would have increased without such contracts.
“In a similar way, QS faculty-student ratio scores of Russian universities that had frequent QS-related contracts increased on 0.9 standard deviations more than they would have increased without frequent QS-related contracts. Taken together, these findings suggest that conflicts of interest may produce significant distortions in global university rankings.”
The senior researcher responsible for the study, Igor Chirikov, said this was the “first empirical evidence to date that rankers’ conflicts of interest may negatively impact outcomes in university rankings”.
“The results are novel for higher education but they are consistent with the numerous studies from the other sectors of economy suggesting that conflicts of interest lead to biased evaluations,” he continued.
“Most likely, rankers that depend on universities for resources are vulnerable to an unconscious self-serving bias also reported by the studies of the conflict of interest among auditors. Self-serving bias may lead to a more favourable consideration of the data coming from universities that are also frequent clients as compared to other types of institutions.”
However, according to Bahram Bekhradnia, president of the Higher Education Policy Institute, the findings should come as little surprise.
“QS is a commercial organisation. They’re there to make money and their rankings are not objective. They sell services to universities to help them to improve their position in the ranking, not to help them improve their quality,” he told The PIE.
“Of course, rankings are not measuring quality, rankings are just measuring whatever they measure. And so we shouldn’t be surprised.”
For Tia Loukkola, European University Alliance director of institutional development, the study is also significant because it reveals a link many in the industry have long suspected but found difficult to prove due to the lack of transparency around which universities use such services.
“It’s an interesting finding. In general this is something that has been discussed, when one part of a company does rankings and the other part brings in money through consulting,” she said.
“There are quite a few universities who use these services. QS has different packages. The data on who uses the services is not publicly available. I think this is part of the solution.
“But the other part is that we need to understand that the rankings and the measures of the rankings are complied by commercial providers. We need to understand how they operate [and] put it into context.”
Rankings have long been criticised from many vantage points, from the influence of consulting services to reliance on reputation to the use of self-reported data.
“I was at the conference in Dubai a couple of years ago and somebody came up to me and told me that they had [self-reported] one measure on the amount of external income. They said that their predecessor had wrongly been reporting the number in the United Arab Emirates dirhams instead of US dollars. So it was inflated by five,” said Bekhradnia.
“The data are not comparable between universities and between countries anyway, let’s face it. And then we got all the shenanigans about selling services. It’s a money making operation and in my view does damage to the university system.”
QS did not respond to requests for comment.
The post Study finds link between uni rankings and QS contracts appeared first on The PIE News.