It may sound silly, but calculating and comparing average class sizes across institutions is very hard to do. Here’s why.
Back when institutions actually paid attention to Maclean’s, the class size questions were the easiest to “massage,” because there was no common definition of what constituted a class. Do course sections count? What about instrument practice classes in music faculties? Many universities gamed the system by including these for the purposes of calculating “average class size” but excluding them when it came to calculating “percentage of classes taught by tenured faculty.”
Playing with numerators and denominators was good for yuks, but not so good in terms of coming up with reliable comparators. Which, of course, was the point. Bluntly, any time you’re counting on institutions to give you accurate data for comparison and some institutions know they won’t come out well in such a comparison, the likelihood increases that said institutions will try to game the data. Way it goes.
But even if the input data were clean, the traditional measure of “average class size” – total student credit hours divided by the number of “classes” (however defined) – leaves much to be desired. Imagine a school with only two classes: one with five hundred students and the other with ten. The “average class size” of this institution using the traditional definition is 255. But this is a vast distortion of reality since 98% of students only experience a class of 500. Matching tiny classes with huge ones can bring the average way down without actually altering the experience of the vast majority of students. In short, the traditional way of measuring class-size can be skewed lower just by adding a few small classes – and it provides significant leeway for institutions to monkey with the data inputs.
But there is there an alternative. Instead of measuring “average class size” (credit hours divided by classes), why not measure “average number of classmates per class”? We’ve been doing it for six years now with the Globe and Mail: asking over 30,000 students each year about each of their classes and specifically asking them how many classmates they have. Using that measure, our fictional two-class university would have a score of 490, which I would argue is probably a better reflection of most students’ experience than the more traditional measure.
It’s a bit crude, to be sure, but the resulting institutional averages are pretty stable over time, which suggests it’s a robust measure. And, crucially, it’s a measure that can’t be gamed. As we’ll see tomorrow when we look at some of the data, the results shed some very interesting light on institutional priorities.
Till then.
Tweet this post
In response to your comments on measuring class size, I must agree. Average class size as defined by anything approximating [ (class size 1) + (class size 2)+ …(class size n) / (number of classes) is too highly affected by outliers. Your suggestion that class sizes averages be determined by asking students how many classmates they have is better powered to capture the experience of the students, which I suggest, is why we have these measures in the first place. I am concerned, however, that it would involve estimates by students who would be notoriously bad at it. Instead, one can design a mathematical way to get at that number. Basically, the numerator would still be a sum of class sizes as recorded by the institution, however the denominator would have to be designed to reflect the number of students who experience that class size. Thus, classes with 200 students would be experienced by 200 students, while classes of 3 would only be experienced by 3 students…and that affect would be represented in the denominator so that the large class sizes are weighted more heavily than the smaller class sizes….allowing the measure of average experience not to be skewed as much by outliers. I’m not sure mathematically how best to do that but I’m thinking that a weighting factor would be appropriate.
You can do the same thing based on admin data. The method doesn’t depend on the use of surveys.
On another discussion forum, I am informed that the math works out to: the sum of the squares of the class sizes divided by the sum of the class sizes; or, more precisely, sigma(class size * [class size – 1])/sigma(class size).
I’d really like someone to try this so we can compare with survey results and see how close the wisdom-of-crowds method we’ve been using comes to nailing it.