Category Archives: why are Toronto students so friggin’ miserable?

December 09

Miserable Toronto Students: Cutting to the Chase

Loyal readers will know we’ve been studying why Toronto students are so miserable for some time now. But we think we’ve found the jackpot here.

Up until now, we’ve mostly been looking at common institutional factors that seem to result in lower satisfaction levels. But it’s time to take a really good look at Toronto students themselves. Could it be that they’re just more demanding/prone to complain/ likely to kvetch? In a word, are they just more crotchety than students from the rest of Canada?

This isn’t an easy theory to test. Ideally, what you’d want is to find out how students across the country feel about some kind of experience that has nothing to do at all with education and see whether Toronto students have a reaction statistically different from the rest of Canada. If Toronto students have similar reactions to students elsewhere, then we might conclude that their bad ratings for universities have to do with the universities themselves. If the ratings turn out differently – that is, if Toronto students are systematically more negative about the unrelated experience as well – then we might chalk up Toronto schools’ perennially ugly results to the fact that their students are just a bit negative/over-entitled/a pain in the ass, etc.

But finding that “something” is tough because you need an experience that will have affected most students across the country in a similar manner. Asking questions about the last South Park episode doesn’t work because not everyone watches it; you can’t ask about price rises at Tim Horton’s because not everyone drinks there; you can’t ask about the weather because it’s always better in Victoria, etc.

But… Facebook to the rescue! Ninety percent of students have Facebook accounts. And Facebook, bless them, performs minor tweaks to its look-and-feel every month or so that inevitably engender a wave of petty, bitter comments from its users. As a result, the question “how annoyed were you with the most recent Facebook changes?” is actually the perfect control for measuring geographical concentrations of student crotchetiness.

We put that question to over 2,000 students across Canada in October through our CanEd Student Research Panel. Here are the proportions of students who said they were “annoyed” or “very annoyed” by the most recent Facebook changes.

Statistically significant at a .005 level, in case you were wondering. Conclusive evidence that there’s a Toronto kvetch factor at work, and something to definitely keep in mind when you look at national satisfaction ratings.

– Alex Usher and Jason Rogers

November 25

Grades, Satisfaction and Miserable Toronto Students

It’s been noted many times (here, for instance) that professors who give easy As tend to do better on course evaluations than those who don’t. But does this work at the institutional level as well?

It’s hard to tell directly because all institutions essentially grade on the same curve. But we can get at it indirectly by looking at the gap between high school and university grades, which does vary significantly – at more selective institutions, students see a drop; at less selective ones their grades tend to get better.

For this analysis we use self-reported data on grades. Now, we know that skeptics say that this is bad methodology because asking students to self-report on grades is like asking men on a dating site to report on their height or income – all three tend to rise in the telling. But what we’re looking at here is change in reported grades. As long as any exaggeration is consistent across time, the exaggerations should cancel each other out. For math-heads, this can be expressed as:

Onwards to look at our sample from the Globe and Mail. We start by arranging the changes in reported grades between high school and university in bands and looking at average satisfaction in each band. It turns out that there is very little change in satisfaction levels unless students see a very large drop in marks (13% or worse).

Figure 1: Satisfaction by Change in Grades from High School to University

Loyal readers will know where this is going. Guess which city has an abnormally high proportion of students whose grades drop precipitously once they get to university?

Figure 2: Percentage of Students with a Drop in Grades of 13% or Worse

How big a difference does this make to satisfaction? Well, check out the differences in satisfaction between students with large grade drops versus others at Toronto institutions; at Mississauga, the difference between students whose grades have fallen a long way and others is greater than one standard deviation.

Figure 3: Average Satisfaction, Students with Large Mark Drop-Offs vs. Others, Toronto

Oddly, when we look at the five institutions elsewhere in Canada with the most students experiencing large drops in marks, we don’t see anything like the same drop in satisfaction, to wit:

Figure 4: Average Satisfaction, Students with Large Mark Drop-Offs vs. Others, Not Toronto

It’s not quite a story about big fish from little ponds getting shocked by the jump to a larger pool. It’s that big fish from Toronto ponds are both likelier to feel a shock and that they feel a whole a lot worse about the jump that fish elsewhere. A simple case of Torontonians’ elevated sense of entitlement? Maybe. Or maybe Toronto is just a more ruthless environment, with higher social penalties for poor performance.

November 11

Why are Toronto Students so Friggin’ Miserable (Part 5)

Last week in our series on student satisfaction (and Toronto students’ lack thereof) we looked at how students’ perception of institutional character – specifically, things like having applied curricula, seeming open to new ideas and offering a supportive environment – correlated with student satisfaction. This week, we’re still on the issue of character, but students’ own characters rather than those of their institutions.

The 2012 Canadian University Report survey asked students how much they agreed, on a one to nine scale, with a series of statements about themselves (e.g. “I am an athlete”). There were eight statements in total, corresponding to “athlete,” “political junkie,” “environmental activist,” “artist,” “technological guru,” “career oriented,” “studious” and “I like to live it up.” (We were unable to ask the more direct question – whether or not students would describe themselves as “liking to party” – because no institution wants to be labelled a “party school” as a result of the CUR).

It turns out that only three of these eight statements have any important relationship to overall satisfaction (Figure 1). The ultimate trifecta of satisfaction? A studious, career oriented student who likes to live it up; the average satisfaction of a student who rates themselves as a nine on all of these measures is 7.5 out of nine.

This, by the way, makes you wonder why any institution would want to avoid being called a party school since such a status is likely to be associated with high levels of satisfaction. To at least some extent, the University of Western Ontario’s continued long-term success in having top satisfaction ratings (which it likes to talk about) is because of its status as a party school (which it would prefer not to talk about). It’s two sides of the same coin.

But back to our longer-term question: does any of this explain why Toronto students are so miserable? Can Toronto institutions’ low satisfaction levels be explained because students there are less likely to describe themselves as “career oriented,” “studious” or “liking to live it up”? Well, maybe a little bit.

Figure 2 shows the percentage of students who rate themselves as an eight or a nine (on a nine-point scale) on each of these three traits. Clearly, Toronto students are less likely to strongly identify with these three traits, but the gap isn’t huge – certainly not enough to explain the big gaps in satisfaction we see each year. That said, it’s worth noting that students at Ryerson – the one Toronto school that does reasonably well on the CUR’s satisfaction measure – are also the ones with the highest average scores for being career oriented and “liking to live it up” and the second-highest average scores for studiousness (behind OCAD).

More next week. Stay tuned.

– Alex Usher and Jason Rogers

November 04

Why Are Toronto Students so Friggin’ Miserable? (Part Quatre)

To date, we’ve been musing about the causes of Toronto students’ dissatisfaction. But let’s put the shoe on the other foot for a bit: what causes student satisfaction to begin with?

One thing we ask in the Globe Canadian University Report survey is students’ perceptions about a number of dimensions of the character of their institution. For instance, we ask them if they think their institution’s curriculum is more theoretical or applied, whether the institution is broadly-based or focuses on a few areas of study, etc. (to save space, we’ll spare you the methodology and wording caveats; contact us directly if you want to know more).

It turns out that students’ perceptions of certain dimensions of institutional character are remarkably closely correlated with satisfaction:

Here’s what Figure 1 tells us: Students are more satisfied if they think their institution isn’t specializing in certain areas, but rather is spreading the (monetary) love about evenly across different fields of study. If they think their curriculum is practical/applied, they’re happier than if they think it’s theoretical. Students who see their schools as expecting them to be self-sufficient or as cautious to new approaches and ideas are miserable; by contrast, those who see their institution as “nurturing and supportive” or “open to new approaches” are extremely satisfied. Indeed, the range of satisfaction from one end of the scale to the other for both of these measures is over three points: that’s larger than the entire range of institutional satisfaction results in the 2012 Canadian University Report.

So, might these factors explain our Toronto problem? Are Toronto institutions seen as too theoretical, closed to new approaches, etc?

It turns out that Toronto institutions aren’t rated differently from institutions elsewhere in terms of spreading their wealth and are barely different in terms of being open to new ideas. Having a theoretical curricula is a big factor for St. George, but not for the rest of Toronto on average (though there is variation – applied curricula at Ryerson and OCAD offset theory at York and the U of T satellites). Only on the issue of being insufficiently nurturing/supportive of students – a measure inversely correlated with institutional size – is there a clear difference between Toronto schools and those elsewhere. But even this is less than it seems. Toronto’s numbers are being driven by the three U of T campuses and OCAD (all of which do badly even once size is taken into account); York’s score is about average while Ryerson does exceptionally well on this measure.

In short, institutional characteristics are an important driver of satisfaction generally, but can only partially explain our Toronto conundrum.

The search continues next week. Stay tuned.

– Alex Usher and Jason Rogers

October 28

Why are Toronto Students so Friggin’ Miserable (Part 3)

So, back to our favourite hobby of delving in to the causes of Toronto students’ misery. Today’ we’re looking at the issue of institutional size and asking the question: are Toronto schools Too Big Not to Fail?

(For those of you tired of hearing about Toronto, bear with us: you can learn a lot about satisfaction generally by following this series.)

First, to put this all in perspective: this year’s Canadian University Report data shows that Toronto students are really unhappy (Figure 1). On a nine-point scale, they rate their satisfaction 0.75 points lower than students from elsewhere in Canada. Given that no school receives an overall satisfaction score less than a 5.8 or greater than 8.2, this is a rather substantial difference (~~a standard deviation of 1.5,~~ 1.5 standard deviations, if your unit of analysis is university means).

One obvious suspect is size. Toronto has some of the largest institutions in Canada, and smaller schools generally do better on the CUR, as a quick glance at the results charts in this year’s edition (or at Figure 2) will tell you.

So, given that a majority of students in our sample attend massive schools like University of Toronto or York University, is “Colossal U” the barrier to Toronto’s satisfaction? A closer look at the data suggests not. Toronto actually has institutions in all four CUR size categories. While the size hypothesis could account for low satisfaction grades at U of T’s St. George campus (B-) or York (C+) (Ryerson’s B actually hugs the national average for schools its size), it hardly explains sentiments at Mississauga (C+) or Scarborough (C+). OCAD’s B-minus is perhaps the ultimate proof; it’s not bad for a Toronto school, but still the lowest score nationally among very small institutions. That’s what teaching Torontonians will do to you.

In short, Toronto’s misery is not concentrated in any one institution or even one type of institution; it’s spread among the big and the small alike, as Figure 3 demonstrates.

So much for that obvious explanation. Maybe the problem is that we’re asking the wrong question: instead of looking at sources of dis-satisfaction, we need to take a harder look at what factors (other than size) are associated with satisfaction – in particular, the correlation between certain types of institutional and student characteristics that seem to positively affect satisfaction. Stay tuned.

– Alex Usher and Jason Rogers.

For the record, no actual Torontonians work at Higher Education Strategy Associates.

October 21

Why are Toronto Students so Friggin’ Miserable? (Part Two)

Today we revisit the issue of why student dissatisfaction in Canada seems to be concentrated in Toronto, aka the Centre of the Universe. We’ll try to answer the simple question – do Toronto schools fare poorly because a disproportionate number of Toronto students live in their parents’ basements?

Our data source today is the HESA-administered survey that fuels the satisfaction results in The Globe and Mail’s Canadian University Report, in which students are asked to express satisfaction on a scale ranging from one (Very Dissatisfied) to nine (Very Satisfied). In practice, students only rarely use the bottom half of the scale, so all institutions receive mean scores greater than five.

Living at home is indeed associated with lower satisfaction (Figure 1). Those who manage to escape their parents’ city entirely are the most satisfied. The difference is small but not insignificant – just over 0.4 points (out of 9) on average between the at-home and away-from-family groups. And Toronto certainly has plenty more kids living at home (Figure 2) – 57% of our Toronto sample lives at home, compared to 33% elsewhere.

Figure 1: Overall Satisfaction with Institution, by Living Arrangement

Figure 2: Living Arrangement by Location

So have we found our answer, then? Well, no. As Figure 3 shows, it’s not quite that simple.

Figure 2: Overall Satisfaction with Institution, by Living Arrangement and Location

While Toronto students who are stuck under the ever-vigilant eyes of their parents are indeed the least satisfied, there remains a large (0.6 points out of 9), significant and unexplained satisfaction gap between this group and those who live at home in other cities. Moreover, there’s still clearly a location effect: students who go away to university in Toronto are less happy than students who stay at home elsewhere in Canada.

So, living at home is clearly part of the answer, but it’s a long way from answering the question of why Toronto students are so friggin’ miserable. Next week, once we have some data from the new Globe survey (new CUR out on the 25th!), we’ll be delving into issues around institutional size and students’ perceptions of institutional mission.

October 03

Why are Toronto Students so Friggin’ Miserable? (Part One)

Seriously, why is it that every time we do a survey for the Globe and Mail, the Toronto schools all look so weak in terms of student satisfaction? The best of the bunch – Ryerson – gets a B+, which is average for the country as a whole, but everyone else regularly gets Bs and B-minuses – or worse.

Objectively, this seems weird. U of T is by any definition a world-class university. York has the challenge of being a commuter school, but new developments to the south of Pond Road are making a difference and some of their new facilities are among the best in the country. Ryerson, now coming up on the 20th anniversary of its shift to university status, is really starting to hit its stride both in terms of research performance and the way it’s handling its physical integration into a rapidly changing downtown core.

So what’s the problem? Is it being in a big city, with all the high costs and long commutes? Hard to see how it could be. UBC, SFU, McGill and Concordia all tend to get higher satisfaction scores than any of the Toronto schools, despite being in similar big-city milieus. Is it institutional size? U of T and York may be behemoths, but that doesn’t seem to fully cut it as an answer. For one thing, it doesn’t explain the poor results at the smaller, cozier, satellite campuses.

It’s a puzzle, all right: but puzzles like that are what we at HESA live for. So we’ve started a mini-research project to explore these questions, and we’ll be sharing the results with you over the next few weeks.

Obviously, we’ll start by taking a deeper look more at the issues like campus size and characteristics. But we’ll also be focusing on the more intriguing possibility that the nature of the student body itself is to blame. With the help of our regular online survey panel, we’ll be testing the proposition that the reason Toronto schools do badly in satisfaction surveys is because Toronto students are just crankier than students elsewhere.

If we’re right, Toronto schools will be able to adjust their results for regional crankiness, and everybody else will feel the kind of smugness they do every time the Leafs miss the playoffs. That’s about as win-win as it gets.

We hope you’ll enjoy the series. Meantime – feel free to send us your own theories.