To continue with the occasional post on random books on the shelves at HESA Towers, today I want to talk about this book pictured below.  It doesn’t look like much, but it’s one of the most intriguing things kicking around the office.  It dates from the summer of 1967 and is called “The Gourman Report”

Most histories of university rankings will tell you that the first commercial ranking of undergraduate institutions was in a 1983 edition of US News & World Report.  There had been lots of rankings before that dating back to the late 1800s, but USNWR’s kind of goofy ranking, which did not yet incorporate any methodology worthy of the name (that didn’t happen until the 1987 edition) is always credited as the first.

Except it wasn’t.  The Gourman Report was.

Over 1100 colleges were listed in the report.  Each got its own page.  At the top of each page was a listing of “key facts” (type of institution, in- and out-of-state tuition, estimated cost of room/board, etc), and in a box under the name of the institution was a three digit score rating the institution on a 200-800 scale, much like the SATs (Princeton came top with a 772, in case you’re wondering).  Here’s the page for Oberlin College.

Now, under the headline stuff came a series of ratings for various aspects of the college’s non-academic functions.  For student services, there were ratings for “counselling”, “scholarships available”, “athletic-academic balance”, and so on.  Under “Administration” there were ratings for “Commitment to Excellence”, “Image of Institution”, “Government Contracts”, and so forth.  Each category generated a rating from A to D, which contributed to the overall score (an A was worth 800, a B 600, etc. – they all summed up to create the institutional averages)

Then each academic department got a rating, A through D, same deal.  In cases where the department in question had existed for less than 10 years, no score was awarded but its presence was denoted to potential students by an X.

This is pretty amazing stuff when you think about it: far more detailed and multi-dimensional than anything that’s been on the market since in North America.  You’d have thought it was a much more acceptable way to do rankings.

Except for one thing: it was pretty much all fake.  The self-published book included a methodology section, but it said nothing about data sources.  There was no survey of institutions to collect data nor, as far as anyone knew, were individuals on campus ever surveyed or contacted for information.  No one ever identified themselves as having worked for Gourman on the project.  And given that there were between 50-120 data points for every institution (probably about 75,000 data points total), that implies Gourman himself – who had a full-time job teaching political science at Cal State Northridge – was himself filling in around 300 datapoints every weekday of the year, which didn’t suggest a whole lot of rigour.

The book went through a number of editions over the years, each with various innovations.  For instance, letter grades were later replaced with an even more ludicrously-detailed three-digit scoring system (did Texas A&M deserve a 2.87 for its German department or a 2.89?).  By the mid-80s, Gourman had even expanded his range to include some of the more famous international universities, including McGill and U of T.  Semi-informed commentators in Japan, who heard about the ranking but weren’t really in on the joke, were horrified by the poor ranking given to the University of Tokyo.  According to Japanese higher education scholar Akiyoshi Yonezawa, it was the shock of this poor ranking which prompted the push for domestic rankings in that country, under the theory that more rigorous scrutiny would force universities to improve.

Eventually, in 1997, the Princeton Review – itself a published of college guides – bought Gourman out, published one final edition of the Report and then mercifully shut down the franchise.  And that was the end of that little chapter of rankings history.  One the industry itself would like you to forget.