[回复本文][原帖] 发信人: DLambda (pe), 信区: forum
标 题: Science 24 August 2007:Who Ranks the University R
发信站: 饮水思源 (2007年08月25日02:24:34 星期六)
Science 24 August 2007:
Vol. 317. no. 5841, pp. 1026 - 1028
Prev | Table of Contents | Next
Who Ranks the University Rankers?
Everyone would like to score well in an academic beauty contest. But is it rea
lly possible to assess an institution's worth?
Who gets to take credit for Albert Einstein's Nobel Prize? The question seems
absurd, but it's important for the reputations of two Berlin universities. The
reason: Even Nobels bagged 90 years ago are counted in the "Shanghai ranking,
" an influential list of the world's 500 best universities. Both Free Universi
ty (FU), founded in West Berlin in 1948, and Humboldt University (HU), on the
other side of the former Wall, claim to be the heirs of the University of Berl
in, the erstwhile home of Albert Einstein and many other Nobelists.
The resulting tug of war has had bizarre results. When the team at Shanghai Ji
ao Tong University produced its first ranking in 2003, it assigned the prewar
Nobels to FU, helping it earn a respectable 95th place. Swayed by protests fro
m the other side of town, the team assigned them to HU in 2004, propelling it
to 95th rank and dropping FU by more than 100 places. After FU in turn cried f
oul--and many e-mails between Germany and China later--the team simply took bo
th universities out of the race. Both are still missing in the 2007 edition, p
ublished 3 weeks ago.
The controversy is just one among many in the booming business of university r
ankings. Invented by the magazine U.S. News & World Report in 1983 as a way to
boost sales, these academic beauty contests--called "league tables" in the U.
K.--now exist at the national level in a dozen countries; there are a handful
of European and global lists as well. Almost all have come under fire from uni
versities, scientists, and, in some cases, fellow rankers.
This year, for instance, presidents of more than 60 liberal arts colleges refu
sed to participate in a key component of the U.S. News & World Report rankings
, published last week. The rankings, they wrote, "imply a false precision and
authority" and "say nothing or very little about whether students are actually
learning at particular colleges or universities." Last year, 26 Canadian univ
ersities revolted against a similar exercise by Maclean's magazine.
The critics take aim not only at the rankings' methodology but also at their u
ndue influence. For instance, some U.K. employers use them in hiring decisions
, says Ellen Hazelkorn of the Dublin Institute of Technology, adding that fund
ing organizations, philanthropists, and governments are paying increasing atte
ntion as well. France's poor showing in the Shanghai rankings--it had only two
universities in the first top 100--helped trigger a national debate about hig
her education that resulted in a new law, passed last month, giving universiti
es more freedom.
So how do you measure academic excellence? Most rankings start by collecting d
ata about each university that are believed to be indicators of quality. After
giving each a different, predetermined "weight," the indicators are added up
to a total score that determines a university's rank. But there are vast diffe
rences in the number and the nature of the indicators, as well as the way the
data are obtained.
National university rankings cater primarily to aspiring students about to cho
ose where to study, which is why they focus on education. In the U.S. News & W
orld Report ranking of "national universities," for instance (there are separa
te lists for many other types of institutions and programs), student retention
rates count for 20%, the average amount spent on each student for 10%, and al
umni donations, believed to reflect student satisfaction, for 5% (see graph).
The University Guide published by the Guardian newspaper in the U.K. has a for
mula with some of the same indicators, but also a 17% weight on graduates' job
SOURCES: U.S. NEWS & WORLD REPORT, TIMES HIGHER EDUCATION SUPPLEMENT, SHAN
GHAI JIAO TONG UNIVERSITY, CWTS
Most international rankings, meanwhile, put a heavy emphasis on research outpu
t. That's in part because they are aimed more at policymakers but also because
education systems and cultural contexts are so vastly different from country
to country that solid and meaningful data are hard to come by. Average spendin
g per student, for instance, doesn't tell you much if you compare China with G
ermany. Nonetheless, the Times Higher Education Supplement (THES) tries to cap
ture education with a few very simple indicators that it believes to be univer
sally valid: the staff/student ratio and the percentages of students and staff
from overseas, regarded as a measure of a school's international cachet.
Ranking education poses another problem: Many rankings rely on universities th
emselves to provide key data, "which is always a deal with the devil," says Al
ex Usher of the Educational Policy Institute Canada in Toronto, who studies ra
nkings. There are documented cases of universities cheating in the U.S. News r
ankings, for instance, and although U.S. News crosschecks the data with other
sources, there are always ways to manipulate them. For example, colleges are k
nown to encourage applications just so they can reject more students, thus boo
sting their score on the "student selectivity" indicator.
Even more controversial are peer-review surveys, in which academic experts jud
ge institutions. THES, for instance, assigns a whopping 40% to the opinions of
more than 3700 academics from around the globe, whereas the judgment of recru
iters at international companies is worth another 10%. But when researchers fr
om the Centre for Science and Technology Studies (CWTS) at Leiden University i
n the Netherlands compared the reviewers' judgments with their own analysis--b
ased on counting citations, an accepted measure of scientific impact--they fou
nd no correlation whatsoever. "The result is sufficient to seriously doubt the
value of the THES ranking study," CWTS Director Anthony van Raan wrote in a 2
The discrepancy might explain why--to the delight of Australian academics and
newspapers--six universities from Australia ended up in the THES top 50 in 200
4, wrote Van Raan, who suspected "strong geographical biases" in the review. M
artin Ince, a contributing editor who manages the THES ranking, says that the
survey has gotten better since 2004 and has a good geographical balance. He be
lieves Australia's strong showing may have been the result of aggressive marke
ting of its universities in Asia. But he concedes that reputation surveys may
favor "big and old universities."
Peer review is also a major bone of contention in the U.S. News ranking. "We g
et a list of several hundred institutions, and we're simply asked to rank them
on a scale of 1 to 5. That's preposterous," says Patricia McGuire, president
of Trinity University in Washington, D.C., and one of those who boycotted the
reputation survey this year. The ranking can't value what her school excels at
, she says: providing a degree to mostly minority women from low-income backgr
U.S. News editor Brian Kelly dismisses the boycott's significance. The ranking
has always had its detractors, he says, but more than half of university offi
cials still fill out the questionnaire. And the magazine could always find oth
er people to review schools.
The Shanghai ranking avoids all of these problems by eschewing university-prov
ided data and expert reviews. Instead, it uses only publicly available data, s
uch as the number of publications in Nature and Science, the number of Nobel P
rizes and Fields Medals won by alumni and staff, and the number of highly cite
d researchers. The result is a list based almost exclusively on research. Nian
Cai Liu, who heads the Institute of Higher Education at Shanghai Jiao Tong Un
iversity, started the ranking 5 years ago because he wanted to know how Chines
e universities were placed in the global pecking order. When colleagues starte
d asking for the data, Liu put them on a no-frills Web site, which now gets th
ousands of visits a day.
But as the Berlin quarrel shows, the ranking has its own problems. For example
, Shanghai credits the institution where the Nobelist worked at the time of th
e award. And that can make a difference. Andrew Fire's 2006 Nobel in physiolog
y or medicine helped his current institution, Stanford University in Palo Alto
, California, move up from third to second place, even though Fire did his gro
undbreaking work on RNA interference while at the Carnegie Institution in Balt
Universities that focus on social sciences or humanities also tend to suffer u
nder the Shanghai system. Recognizing that scientists in those disciplines gra
vitate to different journals, Liu doesn't count Nature and Science papers and
redistributes that 20% share across other indicators. Still, the effect is not
iceable: In 2006, the well-respected London School of Economics and Political
Science ended up in the 201-300 tier (this far down the list, Liu no longer gi
ves individual ranks), whereas the THES awarded the school 17th place.
Well aware of their influence, and the criticisms, the rankers themselves ackn
owledge that their charts aren't the last word. U.S. News & World Report, for
instance, advises students to take many factors into account when choosing a c
ollege. A pop-up window on Liu's Web site warns that "there are still many met
hodological and technical problems" and urges "cautions" when using the result
In response to the critics, some rankers are also continuously tinkering with
their formulas. But that opens them up to another criticism, namely, that a un
iversity can appear to become significantly better or worse in a single year.
Many have accused U.S. News of changing its method precisely to shake up the t
ables and thus boost sales, a charge the magazine rejects.
In part to boost their credibility, the rankers have founded the International
Rankings Expert Group (IREG), which in 2006 came up with a set of ranking gui
delines. Called the Berlin Principles, they stress factors such as the importa
nce of transparency, picking relevant indicators, and using verified data. Ush
er, an IREG member, concedes that the principles are quite general in nature b
ecause they are the "biggest common denominators" among groups of rankers with
very different views. Many rankings aren't fully compliant with the rules yet
, says CWTS researcher Henk Moed.
U = one university in the top 100
* includes Hong Kong (3 universities)
Who's right? Although both agreed that the U.S. led the list and the U.K.
came second, the 2006 Shanghai and THES rankings differed markedly on where Ea
rth's 100 best universities were located.
SOURCE: SHANGHAI JIAO TONG UNIVERSITY, THES
Some believe the way forward lies with more sophisticated ways of presenting t
he data. The group in Leiden, for instance, produces rankings of European univ
ersities based purely on publication and citation data and presents them as no
t one but four tables. Each uses a different variable; the yellow ranking, for
instance, looks at the total number of papers produced, whereas the green ran
king (billed as the "crown indicator") is based on papers' impact, adjusted so
that it doesn't reward bigger institutions or those working in fields in whic
h scientists cite each other more often. The results aren't as simple as a sin
gle list, Moed concedes, but they do provide a more complete picture.
Others are going further. The Centre for Higher Education Development (CHE) in
Gütersloh assesses German university departments without trying to aggregate
them, and the departments are simply slotted into top, middle, and lower tier
s. It also allows the user to sort universities based on their own favorite in
dicators. Obviously, a ranking such as this one makes for less compelling news
paper or magazine copy-like a Miss Universe contest without a winner. But a sp
okesperson for Die Zeit, the German newspaper that publishes the CHE rankings,
says its annual university guide is a bestseller anyway, and the interactive
design lures many readers to its Web site.
With all the complaints, it's easy to forget that rankings have benefits as we
ll, says Moed. Competition spurs universities to actually perform better, he s
ays, and the rankings provide students and policymakers with answers--even if
they're imperfect--to legitimate questions about quality. Other rankers point
out that universities tout the results if they do well, and they don't like be
ing excluded. Perhaps that's why the presidents of the competing Berlin univer
sities announced shortly after the 2007 Shanghai ranking appeared that they wo
uld sit down again to discuss the legacy of Einstein and his illustrious colle
agues. A compromise might propel both back onto the list in 2008.
Einstein might have appreciated the irony. A sign in his office at Princeton r
eportedly read: "Not everything that counts can be counted, and not everything
that can be counted counts."
honda accord crash test video
※ 来源:·饮水思源 bbs.sjtu.edu.cn·[FROM: 22.214.171.124]