Our trustees had lots of questions last fall when they read their copies of the U.S. News & World Report rankings and USA Today’s listing
of the National Survey of Student Engagement. But instead of focusing
on our place on the lists, we advanced our discussion to consider some
promising and meaningful alternatives.
Our committee on academic
affairs and student success asked staff to give us an overview of the
issues. We were especially interested to know whether any of the
rankings had anything to do with measuring how well students were
learning.
Committee members received a report on the criteria and
methodology used by more than 13 organizations that rank colleges. We
learned that these rankings consider everything from faculty salaries to
student ratings of their own schools, to federal survey data on
enrollments, tuition prices, and graduation rates.
Some rankings
organizations do not reveal their methods, and some use questionable
criteria. Indeed, one outfit linked the quality of the institution to
the number, size, and health of the squirrel population on campus! In
another, the ranking correlates directly to the size of the endowment.
We
were given graphs of the University of Tennessee’s performance over the
last five years using broad institutional rankings from U.S. News,
the University of Florida’s Lombardi Program for measuring research
universities, and the Kiplinger.com “best value” college database. We
also had become familiar with specific professional school rankings and Washington Monthly’s
effort to rank colleges and universities by how much they benefit our
country in scientific and social research, service learning, and
fostering social mobility.
Our committee’s conversations produced
the following questions for the board: Which rankings do we want to
track? How much do we spend to participate in these rankings, and how
much time does our institutional research staff spend on this? How
closely aligned is our mission with the ranking criteria? Once the
rankings are published, what do our administrators do with the results?
How does our board use this information for planning?
Perhaps
because our discussions of rankings occurred in the context of the
national debate about how to measure student learning, the most pressing
question boiled down to this: Do we want public officials to be
involved in evaluating educational quality, or should our board be
monitoring the efforts of our faculty and staff to work on their own to
improve assessment and accountability?
Most folks with whom I
have discussed rankings strongly prefer the latter. Hence we were
encouraged this fall by the launch of the Voluntary System of
Accountability. A partnership between the National Association of State
Universities and Land-Grant Colleges and the American Association of
State Colleges and Universities, the VSA intends to provide comparable,
transparent information on the undergraduate experience.
The VSA
is not a new data source; rather, it puts existing data in a useable
format on the Web. It has three primary elements: consumer information
for students and families, current student experiences and perceptions,
and learning outcomes (as measured by a variety of broad and narrow
tests selected by the institutions).
In a similar vein, the
National Association of Independent Colleges and Universities now offers
a Web-based system of learning-outcome measures called University and
College Accountability Network (U-CAN).
Time will tell whether
these new tools will make a difference. But because the federal
government seems more willing than ever to intervene, trustees should
take seriously their responsibility to ensure the institution is setting
appropriately high standards and measuring student learning in a
transparent manner. Measuring student learning must become a part of our
public face.
