HESA

Higher Education Strategy Associates

Category Archives: PSE Outcomes

April 05

Student/Graduate Survey Data

This is my last thought on data for awhile, I promise.  But I want to talk a little bit today about what we’re doing wrong with the increasing misuse of student and graduate surveys.

Back about 15 years ago, the relevant technology for email surveys became sufficiently cheap and ubiquitous that everyone started using them.  I mean, everyone.  So what has happened over the last decade and a half has been a proliferation of surveys and with it – surprise, surprise – a steady decline in survey response rates.  We know that these low-participation surveys (nearly all are below 50%, and most are below 35%) are reliable, in the sense that they give us similar results year after year.  But we have no idea whether they are accurate, because we have no way of dealing with response bias.

Now, every once in awhile you get someone with the cockamamie idea that the way to deal with low response rates is to expand the sample.  Remember how we all laughed at Tony Clement when he claimed  the (voluntary) National Household Survey would be better than the (mandatory) Long-Form Census because the sample size would be larger?  Fun times.  But this is effectively what governments do when they decide – as the Ontario government did in the case of its sexual assault survey  – to carry out what amounts to a (voluntary) student census.

So we have a problem: even as we want to make policy on a more data-informed basis, we face the problem that the quality of student data is decreasing (this also goes for graduate surveys, but I’ll come back to those in a second).  Fortunately, there is an answer to this problem: interview fewer students, but pay them.

What every institution should do – and frankly what every government should do as well – is create a balanced, stratified panel of about 1000 students.   And it should pay them maybe $10/survey to complete surveys throughout the year.  That way, you’d have good response rates from a panel that actually represented the student body well, as opposed to the crapshoot which currently reigns.  Want accurate data on student satisfaction, library/IT usage, incidence of sexual assault/harassment?  This is the way to do it.  And you’d also be doing the rest of your student body a favour by not spamming them with questionnaires they don’t want.

(Costly?  Yes.  Good data ain’t free.  Institutions that care about good data will suck it up).

It’s a slightly different story for graduate surveys.  Here, you also have a problem of response rates, but with the caveat that at least as far as employment and income data is concerned, we aren’t going to have that problem for much longer.  You may be aware of Ross Finnie’s work  linking student data to tax data to work out long-term income paths.  An increasing number of institutions are now doing this, as indeed is Statistic Canada for future versions of its National Graduate Survey (I give Statscan hell, deservedly, but for this they deserve kudos).

So now that we’re going to have excellent, up-to-date data about employment and income data we can re-orient our whole approach to graduate surveys.  We can move away from attempted censuses with a couple of not totally convincing questions about employment and re-shape them into what they should be: much more qualitative explorations of graduate pathways.  Give me a stratified sample of 2000 graduates explaining in detail how they went from being a student to having a career (or not) three years later rather than asking 50,000 students a closed-ended question about whether their job is “related” to their education every day of the week.  The latter is a boring box-checking exercise: the former offers the potential for real understanding and improvement.

(And yeah, again: pay your survey respondents for their time.  The American Department of Education does it on their surveys and they get great data.)

Bottom line: We need to get serious about ending the Tony Clement-icization of student/graduate data. That means getting serious about constructing better samples, incentivizing participation, and asking better questions (particularly of graduates).  And there’s no time like the present. If anyone wants to get serious about this discussion, let me know: I’d be overjoyed to help.

September 14

The Canadian Way of Study Abroad

A few years ago, I think around the time that HESA Towers ran a conference on internationalization, I realized there was something weird about the way Canadian higher education institutions talked about study abroad.  They talked about it as helping students “bridge the gap between theory and practice”, “increasing engagement”, and “hands-on learning”.

That’s odd, I thought.  That sounds like experiential learning, not study abroad.  Which is when it hit me: in Canada, unlike virtually everywhere else in the world, study abroad to a large degree is experiential learning.

In Europe, when they say study abroad, they mostly mean study at a foreign institution in the same field through the Erasmus program.  In the US, they may mean this, or they may means studying in facilities owned by their home universities but located in different countries.  For instance, Wake Forest owns a campus in Venice, Webster University has a campus in Leiden, University of Dallas has one in Rome (have a browse through this list). Basically, if your students are paying megabucks to be at a US campus, the ideas can’t just give them exchange semesters at some foreign public school because who knows about the quality, the safety, etc.

But look at how Canadian institutions showcase their study abroad: McGill talks up its science station in Barbados.  University of Alberta showcases its international internships.  University of Saskatchewan has a fabulous little Nursing programs which ties together practicums in East Saskatoon and Mozambique.  The stuff we like to talk about doesn’t seem to actually involve study in the sense of being in a classroom, per se.  That’s not to say our universities don’t have typical study-abroad programs: we’ve got thousands of those.  They’re just not where the sizzle is.  It’s a distinctly Canadian take on the subject.

This brings me to a point about measuring the benefits of study abroad.  Let’s take it for granted that being abroad  for a while makes students more independent, outward-looking, able to problem-solve, etc.  What is it, exactly, about being abroad that actually makes you that way?  Is it sitting in classes in a foreign country?  Is it meeting foreign people in a foreign country?  Is it meeting people from your own culture in a foreign country (too often the main outcome of study abroad programs)?  What about if you actually get to work in a foreign country? And – crucially for the design of some programs – how long does it take for the benefits to kick in?  A week?  A month?  A year?  When do diminishing returns set in?

Despite study-abroad being a multi-billion dollar niche within higher education, we actually don’t know the answer to many of these questions.  There isn’t a lot of work done which picks apart the various elements of “study abroad” to look at relative impact.  There is some evidence from Elspeth Jones in the UK that many of the benefits actually kick in after as little as 2-4 weeks, which suggests there may be cheaper ways of achieving all these purported benefits.

Of course, one of the reasons we have no answers to this is that it’s pretty hard to unpack the “treatment” involved in study abroad.  You can’t, for instance, randomly assign people to a program that just sits in class, or force people to make friends among locals rather than among the study-abroad group.  But, for instance, it would be possible to look at impacts (using some of the techniques we talked about yesterday) based on length of study abroad period.  It would be possible to compare results of programs that have students mostly sit in class to ones where they do internships.  It would be possible to examine internships based on whether or not they actually made friends among local students or not, a question not asked enough in study evaluation work.  It would also be possible to examine this based on destination country: are the benefits higher or lower depending on proficiency in the destination country’s language?

These questions aren’t easily answerable at the level of an individual institution – the sample size on these would simply be too small.  But one could easily imagine a consortium of institutions agreeing to a common impact assessment strategy, each collecting and sharing data about students and also collectively collecting data on non-mobile students for comparative purposes (again, see yesterday’s blog), perhaps through the Canadian Undergraduate Survey Consortium.  It would make a heck of a project.

If anyone’s interested in starting a consortium on this, let me know.  Not only would it be fun, but it might help us actually design study abroad experiences in a more thoughtful, conscious and impactful way.  And we’d find out if the “Canadian Way” is more effective than more traditional approaches.

Worth a try, I think.

July 25

The low-wage graduate problem

The week before last, the Canadian Centre for the Study of Living Standards (CSLS) put out a report (available hereon trends on low-paid employment  in Canada from 1997 to 2014 (meaning full-time jobs occupied by 20-64 year olds where the hourly earnings are less than 66% of the national median).  It’s an interesting and not particularly sensationalist report based on Labour Force Survey public-use microdata; however one little factoid has sent many people into a tizzy.  Apparently, the percentage of people with Master’s or PhDs who are in low-wage jobs (where the hourly earnings are less than two-thirds of the national median) had jumped from 7.7% to 12.4%.  This has led to a lot of commentary about over-education, yadda yadda, from the Globe and Mail, the CBC, and so on.

This freak-out is a bit overdone. I won’t argue that the study is good news, but I think there are some things going on underneath the numbers which aren’t given enough of an airing in the media.

First of all, as CSLS explains in great detail, the two important findings are that the incidence of low-wage work in the economy has stayed more or less stable, and second, Canadians on the whole are a lot more educated than they used to be.  This leads to a compositional paradox: even though all seven levels of education saw increases in the incidence of low-wages (see Figure below), overall the fraction of Canadians with low wage jobs dropped ever-so-slightly from 27.9% in 1997 to 27.6% in 2014.

ottsyd 20160721-1

Now you have to be careful about interpretation here, particularly with respect to charges of “over-education”.  Yes, the proportion of grads in low-wage jobs is going up.  But the average wage income of university graduates is actually increasing: between 1995 and 2010, it rose by 6% after inflation.  And that’s while the number of people in the labour force with a university degree increased by 94%, and the proportion of the labour force with a university degree jumped from 19.3% to 28.7% (I would break out data on Masters/PhD specifically if I could, but public Statscan data does not separate Bachelors from higher degrees). 

What that tells us is that the economy is creating a lot more high-paying jobs which are being filled by an ever-expanding number of graduates.  But at the same time, more graduates are in low-wage jobs, which suggests that while averages are increasing, so is variance around the mean.

Another factor at work here is immigration.  Since the mid-1990s, the number of immigrants over 25 with university degrees has increased from 815,000 (23.2% of all degree holders) to 1.87 million (33% of all degree holders).  It’s not clear how many of those have graduate degrees (thanks Statscan!) but I think it’s reasonable to assume, given the way our immigration points system works, that the proportion of immigrants with advanced degrees is even higher.

The problem is that immigrants with degrees – particularly more recent immigrants – have a really hard time in the Canadian labour market, particularly at the start (see a great Statscan paper on this here).  To some extent this is rational because the degrees and the skills they confer are genuinely not compatible (see my earlier post on this), and to some extent it reflects various forms of discrimination, but that’s not the point here.  There are over one million new immigrants with degrees over the past fifteen or so years, many of them from overseas institutions.  The CSLS-inspired freak-out is about the fact that over the past 17 years the number of degree-holders has increased by 450,000 (of which 130,000 are at the Master’s/PhD level).  Simple logic suggests that most of the problem people are seeing in the CSLS data is more about our inability to integrate educated immigrants than it is about declining returns to education among domestic students.  I know the data CSLS uses doesn’t allow them to look at the results by where a degree was earned, but I’d bet serious money this is the crux of the problem.

So, you know, chill everybody.  Canadian graduates still do OK in the end.  And remember that comparisons of educational outcomes over time that don’t control for immigration need to be taken with a grain of salt.

May 04

Diverse Sacrifices, Diverse Rewards, Diverse Policies

One of the trickiest things about developing smart higher education policy is that its clients are unbelievably diverse: privileged private-school educated 18 year-olds, first-generation students, working adults, etc.  And the returns to education are equally diverse: strong for Bachelors’ and Master’s Degrees but less so for Doctorates, often strong in professionally-oriented fields and less so in Arts (at least in the first few years).  Coming up with reasonable pricing and student aid policies that can be generally accepted as fair across in the face of all this diversity is a very tricky job indeed.

The first part of this was brought home to me recently when we saw the results of some research  conducted by British Columbia on mature students across Canada.  One of the questions asked was “what’s the biggest sacrifice you have had to make to go back to school”?  The sheer range of answers we got was astonishing.  At one end, there were answers like “I had to give up my gym/yoga membership”, or “I had to give up quinoa” (a high proportion of these, it should be said, came from British Columbia).  The most common response was that people’s social lives were negatively affected because they could no longer afford to eat out with friends.

But at the other end of the spectrum there were some pretty horrific responses.  People who had to pull their kids out of sports teams.  People choosing between rent and food, or rent and medicine.  People who had had spells of homelessness.   All told, the results showed that the several thousand student-aid receiving mature students surveyed, just short of ten percent had experienced a significant form of food or housing precariousness while being a student.

Simply put, there are students who really have very little need of extra help, and there are students who need a lot more help than they currently receive.  This is precisely why the kind of system towards which Canada’s student aid programs are evolving is a good thing: we are withdrawing support from better-off students and concentrating it among worse-off students.  Could we do better?   Sure.  In particular we could do more for the people I call “involuntary students” – people in their 30s or 40s who have cars and houses but who suddenly lose their job and need to re-train.  But the point is, we need more targeted aid, not less.  One-size fits all policies are unhelpful.

It is the same with respect to returns to education.  It is a simple slogan to say that education must be free, that education must not be commoditized.  But it is also a simplistic one.  Low prices (net or sticker) can make a difference in terms of attracting low-income students.  But they also provide huge windfall benefits to students in fields with above-average returns, and it’s really hard to argue that there is any kind of public policy rationale for pricing a public service in such a way that some students (say, in ECE programs) see a very low private return and other students (say, in Dentistry programs) see a very high private return.  There is a way to square this circle: it’s to charge different amounts based on the field of study, and deal with the negative effects of higher fees through income-targeted grants.  Although not all of Canada looks like this, it is more or less the way the system currently works in Ontario. 

The point here is simply this: higher education is not a simple field.  It has many purposes, many clients, many outcomes.  To make it work properly, the policies and regulations which govern it need to be sensitive to this diversity.  Any higher education policy which you can put on a button or a bumper-sticker is therefore likely to be either wrong or wasteful.  

May 02

What’s Going On With College Graduates in Ontario?

I see that Ken Coates and Bill Morrison have just written a new book  called Dream Factories: Why Universities Won’t Solve The Youth Jobs Crisis.  I haven’t read it yet, but judging by the title I’d assume that it makes pretty much the same argument Coates made back in this 2015 paper  for the Canadian Council of Chief Executives, which in effect was “fewer university students, more tradespeople!” (my critique of this paper is here)

With the fall in commodity prices, it’s an odd time to be making claims like this (remember when we had a Skills Gap?  When’s the last time you heard that phrase?).  There’s no evidence based on wages data that trades-related occupations are experiencing greater growth that those in the rest of the economy – since 2007, wages in these occupations have grown at exactly the same rate as the overall economy.  True, occupations in the natural resource sector did experience higher-than-average growth between 2010 and 2014, but unsurprisingly they underperformed the rest of the economy in 2015.  (see figure 1).  More to the point, perhaps, these jobs aren’t a particularly large sector of the economy – if you exclude the mostly seasonal agricultural harvesting category, Canada only has about 265,000 workers in this field.  That’s less than 1.5% of total employment.

Figure 1: Real Wage Increases by Occupation, Canada, 2007-2015, 2007=100

2016-05-01-1

Source: CANSIM

More generally, though, the assumption of Coates and those like him is that in the “new” post-crisis  economy college graduates have qualitatively different (and better) outcomes than university graduates, too.  But a quick look at the actual data suggests this isn’t the case.  Figure 2 shows employment rates 6-months out of college graduates in Ontario over the past decade.  Turns out college graduates have experience more or less the same labour market as university students: an almighty fall post-Lehmann brothers and no improvement thereafter.

Figure 2: Employment Rates of College Graduates, Ontario, 2005-2015

2016-05-01-2

Source: Colleges Ontario Key Performance Indicators

The decline in employment rates can’t really be described as a regional phenomenon, either.  There is not a single college which can boast better employment rates today than it had in 2008: most have seen their rates fall by between 4 and 7 percentage points.  The worst performer is Centennial College, where employment rates have fallen by 13 percentage points; one wonders whether Centennial’s performance has something to do with the very rapid growth in the number of international students it has started accepting in the last decade.

Figure 3: Change in Employment Rates 2008-2015

2016-05-01-3

Source: Colleges Ontario Key Performance Indicators

So what’s going on here?  Is there something that’s changed in college teaching?  Is it falling behind the times?  Well, not according to employers.  Satisfaction rates among employers stayed rock-solid over the period where employment rates fell; and although there has been a slight decline  in the last couple of years, the percentage saying they are satisfied or very satisfied remains over 90%.  Graduate satisfaction fell a bit during the late 00s when employment rates fell, but they too remain very close to where they were pre-crisis.

Figure 4: Employer & Student Satisfaction Rates for College Graduates, Ontario, 2005-2015

2016-05-01-4

Source: Colleges Ontario Key Performance Indicators

My point here is not that colleges are “bad” or universities are “better”.  Rather, my point is that if you measure the success of any part of the post-secondary system exclusively by employment rates, then you’re basically hostage to economic cycles.  Some parts of the cycle might make you look good and others might look bad; regardless, it’s largely out of your hands. So, maybe we should stop focusing so much on this.  And we should definitely stop pretending colleges and universities are different in this respect.

April 06

Fuzzy Skills

About a month ago, Universities Canada held a meeting to talk up the Liberal Arts.  I wasn’t there, and can only go by what I saw on twitter and what I can glean from this University Affairs article which you can read here.  But if the conversation was actually anything like what the sub-head suggests it was (we need better stories!), I’m not impressed.

At one level, “we need better stories” is always true.  Good communication is always worthwhile.  But if you claim that’s all you need then basically you’re saying that actual changes in practices are not necessary. We here in academia are fine, it’s you ignorant lot out there who are the problem – and once we tell better stories, you will see the light.   It’s arrogant, frankly.  More introspection about needed pedgagogical changes and less “we need better stories”, please (I note that Mount Allison’s Robert Campbell at least took that tack – good on him).

Moreover if you look at the “good” stories that Arts faculties want to tell, you’ll find they’re pretty much all about how various social scientists have changed public policy.  Very little is about the humanities (a result perhaps of the usual Canadian confusion about the distinction between “Arts”, “Liberal Arts” and “Humanities”).  At best, you get some vague words about how humanities promotes “soft skills”, which frankly isn’t very helpful.  Partly that’s because “soft skills” as a term is somewhat gendered (and thus likely to turn off males) and partly because there’s very little evidence that humanities education does much to foster that cluster of personality traits, social graces, and all that other stuff which clusters around “emotional intelligence”.  It’s possible – maybe even likely – that humanities graduates might possess these skills, but that may simply be a question of who chooses to enter these fields rather than what skills get developed by the disciplines.

Yet I think there is a simple and unambiguous way to sell the humanities: they are not about soft skills,   they are about “fuzzy skills”.    They are about ambiguity.  They are about pattern recognition.  They are about developing and testing hypotheses in areas of human affairs where evidence is always partial and never clear-cut.  Humanities graduates are not about following rules; they are about interpreting rules when the context changes.  

And you know what?  Doing that kind of interpretation well is *hard*.  The worst mistake the humanities have ever made is accepting the public impression that not being an “exact” science means humanities are “easy”.  They are not.  Good work in the humanities is hard precisely because there are many possible answers to a question.  The difficulty lies in sifting the more plausible from the less plausible (unless of course you dive completely into the post-modernist “I’m OK you’re OK” intellectual rathole where every answer is equally correct; then humanities is just nonsense). 

Think about the world of espionage and intelligence: this is extraordinarily difficult work precisely because we never have enough information and empathy to know exactly what a target is thinking or might be doing.  But it is precisely the synthesis of information from across a wide range of disciplines, and the close reading of texts – what we used to call philology-  that allows us to make competent guesses.  Quantitative data analysis is useful in this (and lord knows we probably shouldn’t let humanities students graduates without some understanding of statistics and probability); but so too are the basic “fuzzy skills” taught in humanities programs.  When business talks about “critical thinking” skills it is precisely this kind of analysis and decision-making, writ small, that they are talking about.

I think that’s a pretty good story for the humanities.  The problem is that for these good stories to work, humanities faculties have to live up to them.  Simply telling a good story isn’t enough. Curricula (and more importantly assessment) need to be re-designed in order to show how these fuzzy skills are actually being taught and absorbed.  No more assuming students get these non-disciplinary skills by osmosis because “everybody knows” that’s what humanities do.  Design for fuzzy skills.  Incorporate them.  Measure them.

And then you’ll have both a good story and a good reality.  That would be real and welcome progress.

January 27

The Future of Work (and What it Means for Higher Education), Part 1

Back in the 1990s when we were in a recession, Jeremy Rifkin wrote a book called The End of Work, which argued that unemployment would remain high forever because of robots, information technology, yadda yadda, whatever.  Cue the longest peacetime economic expansion of the century.

Now, we have a seemingly endless parade of books prattling on about how work is going to disappear: Brynjolfsson and McAfee’s The Second Machine Age, Martin Ford’s Rise of the RobotsJerry Kaplan’s Humans Need not Apply, Susskind and Susskind’s The Future of the Professions: How Technology will Transform the Work of Human Experts (which deals specifically with how info tech and robotics will affect occupations such as law, medicine, architecture, etc.), and from the Davos Foundation,  Klaus Schwab’s The Fourth Industrial Revolution. Some of these are insightful (such as the Susskinds’ effort, though their style leaves a bit to be desired); others are hysterical (Ford); while others are simply dreadful (Schwab: seriously, if this is what rich people find insightful we are all in deep trouble).

So how should we evaluate claims about the imminent implosion of the labour market?  Well first, as Martin Wolf says in this quite sober little piece in Foreign Affairs, we shouldn’t buy into the hype that “everything is different this time”.  Technology has been changing the shape of the labour market for centuries, sometimes quite rapidly.  We will go on changing.  The pace may accelerate a bit, but the idea that things are suddenly going to “go exponential” are simply wrong.  Just because we can imagine technology creating loads of radical disruption doesn’t mean it’s going to happen.  Remember the MOOC revolution, which was going to wipe out universities?  Exactly.

But just because the wilder versions of these stories are wrong doesn’t mean important things aren’t happening.  The key is to be able to lose the hype.  And to my mind, the surest way to get past the hype is to clear your mind of the idea that advances in robotics or information technology “replace jobs”.  This is simply wrong; what they replace are tasks.

We get a bit confused by this because we remember all the jobs that were lost to technology in manufacturing.  But what we forget is that the century-old technology of the assembly line had long turned jobs into tasks, with each individual performing a single task, repetitively.  So in manufacturing, replacing tasks looked like replacing jobs.  But the same is not true of the service sector (which covers everything from shop assistants to lawyers).  This work is not, for the most part, systematic and routinized, and so while IT can replace tasks, it cannot replace “jobs”  per se.  Jobs will change as certain tasks get automated, but they don’t necessarily get wiped out.  Recall, for instance, the story I told about ATMs a few months ago: that although ATMs had become ubiquitous over the previous forty years, the number of bank tellers not only hadn’t decreased, but had actually increased slightly.  It’s just that, mainly, they were now doing a different set of tasks.

Where I think there are some real reasons for concern is that a lot of the tasks that are being routinized are precisely the ones we used to give to new employees.  Take law, for instance, where automation is really taking over document analysis – that is, precisely the stuff they used to get articling students to do.  So now what do we do for an apprenticeship path?

Working conditions always change over time in every industry, of course, but it seems reasonable to argue that job change in white-collar industries – that is, the ones for which university education is effectively an entry-requirement – are going to change substantially over the next couple of decades.  Again, it’s not job losses; rather, it is job change.  And the question is: how are universities thinking through what this will mean for the way students are taught?  Too often, the answer is some variation on “well, we’ll muddle through the way we always do”.  Which is a pretty crap answer, if you ask me.  A lot more thought needs to go into this.  Tomorrow, I’ll talk about how to do that.

November 19

Stories Arts Faculties Tell Themselves

Here at HESA towers, we’ve been doing some work on how students make decisions about choosing a university (if you’re interested: the Student Decisions Project was a multi-wave, qualitative, year-long longitudinal study that tracked several hundred Grade 12 students as they went through the PSE research, application, and enrolment process.  We also took a more targeted qualitative look, specifically at Arts, with the national Prospective Arts Students Survey).  We’ve been trying to do the same for colleges, but it’s a much trickier demographic to survey.

In both studies, one of the questions we asked is what students really want from their education.

Now at one level, this question is kind of trite.  We know from 15 years of surveys from the Canadian Undergraduate Survey Consortium that students go to university: i) to get better jobs; ii) because they like learning about a particular field; and also, iii) to make friends, and enjoy the “university experience”.

Where it gets a little trickier, however, is when you break this down by particular fields of study.  With most faculties, there tends to be a positive reason to attend.  However, when it comes to Arts, enrolment is often seen as a fall-back option – it’s something you do if you don’t have concrete goals, or if you can’t do anything else.  Now, Arts faculties tend to take the positive here, and spin this as students wanting to “find themselves”. But in deploying this bit of spin, Arts faculties often end up heading in the wrong direction.

One of the problems here is that the notion of students “finding themselves” (not a term students themselves use) is not as straightforward as many think. Broadly, there are three possible definitions.  The first situates “finding yourself” in academic terms: by exploring a lot of different academic options, a student finds something that interests her/him, and becomes academically engaged.  This is one of the reasons that Arts faculties are built around a smorgasbord model, which lets students “taste” as many things as possible, and hence “discover” themselves.

But that’s not the only possible definition of “finding oneself”.  There is another option, in which students essentially view PSE as a cooling out period where they can “find” what they want to do, in a vocational sense.  Yes, they are taking courses, but since they recognize that Arts courses don’t lead directly to employment, they are more or less marking time while they discover how to make their way in the employment world, and think about how and where they want to live.  Then there is a third, slightly different take, in which students view “finding themselves” as the process by which they acquire transversal skills, and the skills of personal effectiveness needed to be successful adults.  School is something they do while they are learning these skills, often for little reason other than that going to school is something they have always done, and in many cases are expected to do.

Though all of these interpretations of “finding yourself” have some currency among students, it probably shouldn’t come as a surprise to learn that the one about “finding yourself” being a voyage of academic discovery is, in fact, the least frequently mentioned by incoming students.  Now, maybe they come around to this view later on, but it is not high on the list of reasons they attend in the first place.  To the extent that they have specific academic interests as a reason for enrolling in Arts, they tend to be just that: specific – they want to study Drama, or History, or whatever.

Which raises two questions: if this is true, what’s the benefit of Arts faculties maintaining such a wide breadth of requirements?  And second, why aren’t Arts faculties explicitly building-in more transversal skills elements into their programs?  Presumably, there would be a significant advantage in terms of recruitment for doing so.  Someone should give it a whirl.

November 16

An Interesting but Irritating Report on Graduate Overqualification

On Thursday, the Office of Parliamentary Budget Officer (PBO) released a report on the state of the Canadian labour market.  It’s one of those things the PBO does because the state of the labour market drives the federal budget, to some extent.  But in this report, the PBO decided to do something different: it decided to look at the state of the labour market from the point of view of recent graduates, and specifically whether graduates are “overqualified” for their jobs.

The methodology was relatively simple: using the Labour Force Survey, determine the National Occupation Code (NOC) for every employed person between the ages of 25 and 34.  Since NOCs are classified according to the level of education they are deemed to require, it’s simple to compare each person’s level of education to the NOC of the job they are in, and on that basis decide whether someone is “overqualified”, “underqualified” or “rightly qualified”.

So here’s what the PBO found: over the past decade or so, among university graduates, the rate of overqualification is rising, and the rate of “rightly qualified” graduates is falling.  Among college graduates, the reverse is true.  Interesting, but as it turns out not quite the whole story.

Now, before I get into a read of the data, a small aside: take a look at the way the PBO chose to portray the data on university graduates.

Figure 1: Weaselly PBO Way of Presenting Data on Overqualification Among 25-34 Year Old University Graduates

1

 

 

 

 

 

 

 

 

 

 

 

 

 

Wow!  Startling reversal, right?  Wrong.  Take a look at the weaselly double Y-axis.  Here’s what the same data looks like if you plot it on a single axis:

Figure 2: Same Data on University Graduate Overqualification, Presented in Non-Weaselly Fashion

unnamed

 

 

 

 

 

 

 

 

 

 

 

 

See?  A slightly less sensational story.  Clearly, someone in PBO wanted to spice up the narrative a bit, and did so by making a pretty unforgivable graph, one that was clearly meant to overstate the fundamental situation.  Very poor form from the PBO.

Anyways, what should we make of this change in university graduates’ fortunes?  Well, remember that there was a massive upswing in university access starting at the tail end of the 1990s.  This meant a huge change in attainment rates over the decade.

Figure 3: Attainment Rates Among 25-34 Year-Olds, Canada

unnamed

 

 

 

 

 

 

 

 

 

 

 

 

What this upswing in the university attainment rate meant was that there were a heck of a lot more university graduates in the market in 2013 than there was, say, a decade earlier.  In fact, 540,000 more, on a base of just over a million – a 53% increase between 1999 and 2013.  Though the PBO doesn’t mention it in the report, it’s nevertheless an important background fact. Indeed, it likely explains a lot of the pattern change we are seeing.

To see how important that is, let’s look at this in terms of numbers rather than percentages.

Figure 4: Numbers of Rightly-Qualified and Overqualified 25-34 Year Old University Graduates, 1999-2013

1

 

 

 

 

 

 

 

 

 

 

 

 

In fact, the number of rightly-qualified graduates is up substantially over the last decade, and they’ve been increasing at almost (though not quite) as fast a rate as the number of “overqualified” graduates.  For comparison, here’s the situation in colleges:

Figure 5: Numbers of Rightly-Qualified and Overqualified 25-34 Year Old College Graduates, 1999-2013

unnamed

 

 

 

 

 

 

 

 

 

 

 

 

As advertised, there’s no question that the trend in college outcomes looks better than the one for universities.  Partly that’s because of improvements in colleges’ offerings, and partly it has to do with the run-up in commodity prices, which made college credentials more valuable (remember the Green-Foley papers? Good times).

What should you take from all of this?  If nothing else, don’t forget that comparing university outcomes over time is hard because of the changing size and composition of the student body.  Remember: the median student today wouldn’t have made it into university 25 years ago.  Average outcomes were always likely to fall somewhat, both because more graduates means more competition for the same jobs, and also because the average academic ability of new entrants is somewhat lower.

It would be interesting, for instance, to see these PBO results while holding high school grades constant – then you’d be able to tell whether falling rates of “rightly-qualified” graduates were due to changing economy/less relevant education, or a changing student body.  But since we can’t, all one can really say about the PBO report is: don’t jump to conclusions.

Especially on the basis of those godawful graphs.

October 15

The Most Horrifying Book of the Year

One of the most famous studies on higher education and opportunity was published a little over fifteen years ago by economists Alan Krueger and Stacy Berg Dale.  Using something called the College and Beyond Survey, they followed over 6,000 students who had been accepted to American universities in 1976, and then looked at their outcomes almost twenty years later, in 1995.  The key finding was that holding SATs constant, school selectivity didn’t matter much.  The important thing wasn’t attending Harvard, it was having the marks to get into Harvard (for whites, at least – for black students, accessing the networks available to selective school alumni did have a positive effect on education).

As selective universities became even more selective during the early 2000s, and their average student’s socio-economic background drifted upwards, this study was comforting: the rich were just wasting their money on Ivies, and the poorer but talented who were excluded from those schools would end up doing just as well, even if they went elsewhere.  Except: I’ve never met an American parent who actually believed the study.  Sure, they’d say, but things have changed since the mid-70s.  Networks matter more, and by God the one thing you get at an Ivy League school is a network.

A book came out this past spring that explains why they’re right and Kruger and Dale are, if not wrong, then at least out of date.  It’s called Pedigree: How Elite Students Get Elite Jobs by Lauren Rivera, a professor at the Kellogg School of Business at Northwestern University.   It follows the graduate recruiting activities of a number of unnamed banking and consulting firms – the tiny fraction of companies that give 22 year-olds jobs that pay roughly six figures in the first year – across a number of Ivy League schools. And it will horrify you, and make you profoundly glad you don’t live in the United States.

The first couple of chapters are pretty mind-blowing.  The descriptions of how these major banks and consulting companies are dropping hundreds of thousands of dollars – each – on recruiting activities on these Ivy League campuses are bad enough (there must be a lot of very good parties at these places). What is truly mind-blowing is that all of these organizations, filled with “masters of the universe” types, genuinely seem to think that whatever Harvard and Princeton admissions is doing, it must be absolutely, 100% right, because that’s effectively the only filter they have.  If you’re from a selective Ivy, these purveyors of elite jobs think “they must be smart and fast learners; let’s hire them”.  From a state school?  You don’t even get an interview.  What you have accomplished in your four years of higher education is irrelevant; it’s all about where you went to school.

Not only is this mindbogglingly stupid (and, to tell the truth, odd – surely one of these companies should have arbitraged this over-focus on a tiny minority, and gone big on flagship state university hiring?), but also it is infuriating in the extreme.  Read it, or at lest the first couple of chapters, which describe how firms filter by campus (it gets less interesting after that): if your blood doesn’t boil, you have no soul.

A last note: try reading this book next to Kevin Carey’s The End of College, or Ryan Craig’s College Disrupted.  Both Carey and Craig are of the opinion that one thing that will drive disruption in higher education is that companies are going to want the “best” employees. Eventually, they say, algorithms are going to come along and find those employees wherever they are, at which point the prestige value of a degree from a selective school will disappear quickly.

But reading Pedigree it becomes clear that what employers are usually looking for are people who remind them of themselves.  People they won’t mind hanging out with on a long night in the office.  People who will fit in.  And you don’t need an algorithm for that.  You just need an old boys’ club.

Page 1 of 612345...Last »