Ranking the college rankings: What should they actually tell us?

The Hechinger ReportSeptember 25, 2013 

— President Barack Obama wants Washington to grade colleges and universities based on their costs and success rates, but some people are already way ahead of him.

At a time when students and their families are demanding to know what they’re getting for their mounting investments in higher education, several foundations and research centers already are working on new ways to show them.

Some schools – reasoning that it’s better to come up with their own ratings than have them imposed by someone else – are even quietly developing new ways to gauge what graduates learn and earn, though many remain reluctant so far to make the results public.

“One thing everyone seems to agree on is that we should have a good way for people to choose where to go to college,” said Zakiya Smith, strategy director at the Lumina Foundation, a private group that promotes higher education.

The foundation is offering $10,000 in a crowd-sourced competition to come up with the best way to make more user-friendly a White House website called the College Scorecard.

Obama has proposed that the government publicly rate colleges and universities by 2015, based on average student debt, graduation rates, graduates’ earnings and other barometers.

“The answers will help parents and students figure out how much value a college truly offers,” the president said in a speech last month at the University of Buffalo.

That’s information consumers increasingly want. In a survey released in January by Hart Research Associates, 84 percent supported the idea of making colleges disclose information about graduation and job-placement and loan-repayment rates.

“People are looking at, ‘Where do we get the biggest bang for our buck?’” said Terrell Halaska, a partner at HCM Strategies, a higher education consulting firm. “They’re desperately looking for high-quality consumer information. They don’t know where to turn. There are 1,000 different ranking systems out there.”

Universities are skeptical that the government should add yet another one. But some are privately working on their own ratings systems.

With money from the Bill & Melinda Gates Foundation, 18 higher education institutions have been at work on something called the Voluntary Institutional Metrics Project. Coordinated by HCM, it would provide college-by-college comparisons of cost, dropout and graduation rates, post-graduate employment, student debt and loan defaults, and how much people learn. (Gates and Lumina are among the funders of The Hechinger Report, which produced this story.)

But after two years, the group still hasn’t figured out how to measure what is, after all, the principal purpose of institutions of higher education: Whether the people who go to them actually learn anything, and, if so, how much?

The many existing privately produced rankings, including the dominant U.S. News & World Report annual “Best Colleges” guide, have historically rewarded universities based on the quality of the students who select them, and what those students know when they arrive on campus – based on their SAT scores, class rank and grade-point averages – rather than what they learn once they get there.

U.S. News has been gradually shifting toward incorporating in its rankings such “outputs” as graduation rates, the publisher says.

Still, the most popular rankings “have been almost completely silent on teaching and learning,” said Alexander McCormick, director of the National Survey of Student Engagement, another attempt by universities to measure their own effectiveness.

And that, he said, is “like rating the success of hospitals by looking only at the health of their patients when they arrive.”

The National Survey of Student Engagement, which is based at the Indiana University School of Education, seeks to change that calculation. Each spring, it surveys freshmen and seniors at as many as 770 participating universities and colleges about their classroom experiences, how much they interact with faculty and classmates, whether their courses were challenging, and how much they think they’ve learned.

But the project also spotlights a big problem with potentially valuable ratings collected by the institutions themselves: The schools are often unwilling to make them public.

So while it was conceived in 2000 with great fanfare as a rival to the U.S. News rankings, the National Survey of Student Engagement remains obscure and largely inaccessible. The results are given back to the participating institutions, and while a few schools make some of them public, others don’t, thwarting side-by-side comparisons.

There are other drawbacks to self-rating.

One is that the information is self-reported, and not independently verified, potentially inviting manipulation of the figures.

In the last two years, seven universities and colleges have admitted falsifying information sent to the Department of Education, their own accrediting agencies, and U.S. News: Bucknell University, Claremont McKenna College, Emory University, George Washington University, Tulane University’s business school, and the law schools at the University of Illinois and Villanova University.

Still, there’s evidence that students and their families don’t rely as much on rankings as university administrators seem to fear. Rankings are a mediocre 12th on a list of 23 reasons for selecting a college that students gave in an annual survey by the University of California, Los Angeles, Higher Education Research Institute.

“People appreciate information,” said Smith of the Lumina Foundation. “When you buy a car, a lot of things may come into consideration, but you still want to know what the gas mileage is. And you have the right to know.”

The State is pleased to provide this opportunity to share information, experiences and observations about what's in the news. Some of the comments may be reprinted elsewhere in the site or in the newspaper. We encourage lively, open debate on the issues of the day, and ask that you refrain from profanity, hate speech, personal comments and remarks that are off point. Thank you for taking the time to offer your thoughts.

Commenting FAQs | Terms of Service