HESA

Higher Education Strategy Associates

August 31

Who’s Not in the U-15 (But Could Be)

One of the interesting things about our new research rankings – which unlike previous attempts at such things are fully field-normalized – is that it shines a very different light on who the “leaders” are in terms of research.

Back in the day, the ten “leading” research institutions in the country (Laval, McGil, Montreal, Queen’s, Toronto, McMaster, Waterloo, Western, Alberta and UBC) created the “G-10.” It was a talking-shop, mostly: a forum where big universities could exchange data quietly amongst themselves. Around the turn of the century, three more institutions (Ottawa, Calgary and Dalhousie) were added, and more recently Manitoba and Saskatchewan were included as well.

Waterloo apart, the U-15 is basically a list of the country’s established medical schools. But the idea that simply having a medical school makes you research-intensive is questionable. If you were looking for “research leaders,” you’d probably start with looking at bibliometric measures, like the H-index measures. There are 16 schools which have an average H-index score above one (i.e., where the average professor at that school has an H-index above the national average) in NSERC disciplines, and 22 which have an average H-index above one in SSHRC disciplines.

So how does the U-15 membership fare in these? UBC, Toronto, McGill and Montreal and are in the top five in both SSHRC and NSERC disciplines, so they’re indisputably “tops.” Queen’s, McMaster, Alberta, Waterloo and Manitoba all have above-average scores in both areas. After that, it gets trickier: Ottawa has a well-above average score in NSERC disciplines but a below-average one in SSHRC disciplines; Saskatchewan and Calgary are above-average in SSHRC disciplines but not in NSERC ones. Laval, Dalhousie and Western are below-average in both.

But what about schools outside the U-15? Well, Simon Fraser makes the top ten in both fields, a claim most of the U-15 can’t make. York, Concordia and Trent (yeah, we did a double-take, too) both have above-average scores in both fields; from a purely bibliometric perspective, they are at least in the same class as Manitoba. Trent and Concordia don’t look so good when funding measures are taken into account, but the other two do OK and seem at least the equal of a few of the U-15. One could also make a decent case for Guelph, which is well above average in SSHRC disciplines, and only a shade below it in NSERC ones.

So why aren’t these schools in the U-15’s big research club, even though they clearly outperform some of the weaker U-15 members? Unfortunately, the answer is prestige. If York and Concordia were allowed into the club, Toronto and McGill would probably want to find another sandbox to play in. In academia, exclusivity matters.

August 30

Research Rankings: Burning Questions

We understand that some results from our research rankings are causing some head-scratching. We thought we’d give you some insight into some of the key puzzles.

Q: Why isn’t U of T first? U of T is always first.

The fact that we didn’t include medical research is a big reason; had we done so, the results might have been quite different. But part of it also is that Toronto’s best subjects tend to be ones with high research costs and high publication/citation rates. Once you control for that, UBC surpasses Toronto on all measures.

Q: Why does UBC appear to be so much better than everyone else in SSHRC-related disciplines?

A variety of reasons, but much of it is down to the fact that the Sauder School is really good.

Q: Looking at the data, which schools stand out as being under-rated?

Simon Fraser makes the top ten in both SSHRC and NSERC disciplines, which most of the U-15 can’t say. UQ Rimouski came seventh in science and engineering – they aren’t very big but their strength in marine sciences puts them close to the top overall. In SSHRC-related disciplines, the answer is Guelph, which does extremely well in this area, despite having a reputation which is more science-based. York and Trent over-perform in both science and arts. York might not be such a surprise – it’s a big school with lots of resources even if it isn’t super in any of the “money” disciplines. But Trent was a revelation – by far the best publication record of any small-ish school in the country across all disciplines.

Q: And over-rated?

Despite being U-15 members, Western, Dalhousie and Laval all had relatively modest performances. At these schools more than the others, a lot of their research prestige seems to hang on their medical faculties.

Q: Any anomalies?

Apart from l’Université de Montréal, none of the francophone schools do very well in the social sciences and humanities rankings, and the culprit is on the bibliometric side rather than the funding side. The practice of publishing in French has the tendency to lessen the size of the potential audience. This reduces potential citations and hence reduces H-index scores. In the sciences and engineering, where publication tends to happen in English, francophone schools punch actually punch above their weight.

Q: Any trends of note?

UBC aside, its’ the Ontario institutions who really steal the show. Sure, they’re funded abysmally, but they perform substantially better on publication measures than anyone else in the country. We can’t say why, for sure, but maybe those high salaries really work. They’re tough on undergrad class sizes, though…

August 29

Research Rankings

Today, we at HESA are releasing our brand new Canadian Research Rankings. We’re pretty proud of what we’ve accomplished here, so let me tell you a bit about them.

Unlike previous Canadian research rankings conducted by Research InfoSource, these aren’t simply about raw money and publication totals. As we’ve already seen, those measures tend to privilege strength in some disciplines (the high-citation, high-cost ones) more than others. Institutions which are good in low-citation, low-cost disciplines simply never get recognized in these schemes.

Our rankings get around this problem by field-normalizing all results by discipline. We measure institutions’ current research strength through granting council award data, and we measure the depth of their academic capital (“deposits of erudition,” if you will) through use of the H-index, (which, if you’ll recall, we used back in the spring to look at top academic disciplines). In both cases, we determine the national average of grants and H-indexes in every discipline, and then adjust each individual researcher’s and department’s scores to be a function of that average.

(Well, not quite all disciplines. We don’t do medicine because it’s sometimes awfully hard to tell who is staff and who is not, given the blurry lines between universities and hospitals.)

Our methods help to correct some of the field biases of normal research rankings. But to make things even less biased, we separate out performance in SSHRC-funded disciplines and NSERC-funded disciplines, so as to better examine strengths and weaknesses in each of these areas. But, it turns out, strength in one is substantially correlated with strength in the other. In fact, the top university in both areas is the same: the University of British Columbia (a round of applause, if you please).

I hope you’ll read the full report, but just to give you a taste, here’s our top ten for SSHRC and NSERC disciplines.

Eyebrows furrowed because of Rimouski? Get over your preconceptions that research strength is a function of size. Though that’s usually the case, small institutions with high average faculty productivity can occasionally look pretty good as well.

More tomorrow.

August 28

Research Grants by Discipline

So, tomorrow, HESA will be releasing its inaugural set of Canadian research rankings. We think they’re pretty cool; not only are they the first attempt in Canada to employ field-normalization techniques on bibliometric data, as far as we’re aware, they’re the first rankings anywhere in the world to employ field-normalization on research income.

Why does this matter? Well, not all research was created alike. Each discipline has a different publication culture, for starters. The average H-index score for an academic in astrophysics is about four times that of an environmental scientist and ten or eleven times that of a historian. Without field-normalization, any mediocre bunch of physicists trounces the best history department in the world. Yet, remarkably, most rankings and ratings systems choose to compare universities without normalizing for differences in publication culture.

It’s the same with research funding. Not only are researchers in some disciplines likelier to receive money than others, but the size of the average grant also differs because it’s inherently more expensive to run experiments in some disciplines than others. The gap between disciplines would be even greater if NSERC actually funded projects fully, but that’s another story.

A few months ago, we showed you some of the differences in disciplinary H-index averages, so you should already have a sense of how those differences play out. But we haven’t shown you the differences in funding by discipline. And so, herewith, the average amount of granting council funds distributed in 2010-11 per professor, by discipline, for selected disciplines.

Average Granting Council Awards per Faculty Member, by Discipline

There aren’t a lot of real surprises here: on average, the amount of funding per faculty member in engineering is about sixteen times what it is in the humanities and seven times what it is in the social sciences. It’s also about 40% more than it is in the sciences. This, of course, is the reason why one should field-normalize data; without it, schools with large engineering schools will tend to look good regardless of how good their scholars are in the rest of the university.

Anyways, all of this and more tomorrow in our all new rankings! (You know you love them, you naughty people.)

August 27

Follies of the Rich and Philanthropic

One of the irritating things about modern capitalism is the way media and the establishment see the need to treat rich people’s ideas seriously, even if the subject at hand is something they know literally nothing about.

Take, for instance, a recent story in Businessweek about Chamath Palihapitiya. A former Facebook insider, he’s now a venture capitalist with an eye for social entrepreneurship. The story has him creating a company called Brilliant, a “global online talent registry” that plans to identify “the world’s smartest kids” (the implication in the story is that it’s aimed at overlooked geniuses in developing countries) and then fund their way into university for a share of their lifetime income (presumably on the model of companies like Lumni and My Rich Uncle.

The assumptions that go into this idea are plain barmy. First, it assumes that developing-country education systems can’t identify geniuses on their own and second, it assumes that there is a way to identify such people through a set of on-line tests. The fact is, even if one could create a better set of aptitude tests than the ones ETS has spent hundreds of millions of dollars developing over the last 60 years (and work out the massive invigilation issues involved), the likelihood is that an overlooked genius in a poor country has never been taught the skills that would allow them to succeed on such a test. But Palihapitiya’s rich, so this flawed venture gets laudatory coverage.

Or, take Seymour Schulich’s recent foray into scholarships. Though many of Schulich’s philanthropic investments have been excellent, his attempt to create a “Canadian Rhodes” (i.e., very prestigious scholarship) was not. Prestige does not, as Schulich seems to think, stem from giving out the biggest wad of cash. Bill Gates gave Cambridge an enormous whack of money to create the Gates Scholarship program, with stipends twice as large as the Rhodes, but does anyone give think of Gates Scholars as being in the same class as Rhodes Scholars? No, because the Rhodes prestige is based on a unique and rigorous selection criteria and process which has delivered over a century of distinguished alumni, not cash. Got that? Criteria, process, track record, with the third highly dependent on the first two.

So what does Schulich do? He farms out the criteria and process pieces to the United Jewish Appeal. I’m sure they are very nice people, but with zero experience in this area, the odds of them getting this right first time are pretty long, and there goes the Rhodes thing. But since Schulich is rich, no one told him he was making a poor decision.

God bless people in advancement. I sure couldn’t put up with that stuff.

August 24

The Road to Three

Glenn Murray is a man in a hurry. He talks – it’s never clear how seriously – about shortening degrees to three years within the lifetime of this government. Let’s be generous and grant that the McGuinty government will actually last a full four years – what are the odds of getting to achieving this?

Honestly? Zero. Zip. Bupkis. Here’s why:

There are only two feasible routes to three-year degrees – the compression model and the re-design model. The former is superficially a lot more palatable, in the sense that it doesn’t force profs to re-design curricula and can be implemented more quickly. But in practice, it’s not clear if this is actually the more workable path. The main barrier to this approach is that it still requires a major change in student behavior. And that isn’t going to happen without incentives.

Incentives could be made direct to students by improving student assistance. That sounds simple, if potentially expensive (this is possibly a deal-breaker, given that saving money is job one for Murray). But since only 50% of students use OSAP, half the student body would be unaffected. The only truly effective way to introduce incentives across the entire student body is to change the fee structure, giving students rebates for finishing early or charging penalties for finishing late. But that requires institutions to play ball. So maybe it’s less a matter of incentivizing students than of incentivizing institutions; tweak the funding formula to favour institutions that gets students out in three years, and let institutions themselves work out how best to get students to achieve it. But since most of the economic gains come from larger class sizes and increased student aid costs offset much of the purported gains – can anyone really see the premier sticking his neck out to annoy the universities that way?

The alternative is re-design. This approach has the potential at least to save money and make the system more effective – but even assuming everyone buys into it, drastic curriculum re-design isn’t quick and requires extensive pilot testing. The best analogy here is Ontario’s reducing secondary school from five years to four. This became mandatory in 2002, but was preceded by a fourteen-year period in which four-year graduation was optional. This in turn was preceded by a multi-year period of curricular adjustment; in total, it was almost twenty-five years from the time people began re-designing the curriculum for optional four-year graduation to the time it became compulsory.

I can imagine it being faster for universities; for all their alleged slowness, where curriculum is concerned they change more quickly than secondary schools. But we’re still talking decades rather than years.

Someone needs to tell the Minister.

August 23

Re-designing to Three

So, we’ve covered the ideas of cutting graduation requirements, bringing back grade 12, and degree compression as ways to get to a three-year degree. That leaves course re-design.

There are some examples out there of full-on re-design of programs from four years of seat time to three years of competencies. The best known is probably at Southern New Hampshire University (SNHU), which was the subject of a recent book called Saving Higher Education: The Integrated, Competency-Based Three-Year Bachelor Degree Program. The basics of program re-design are relatively straightforward – working backward from a statement of graduate competencies (outcomes, basically), you work out how to introduce, reinforce and perfect those competencies in the space of 3 normal academic years. SNHU went whole-hog on this, even replacing “courses” with non-concurrent “modules,” hence ditching the idea that all academic units need to be of one term’s duration. Few, in fact, last more than six weeks (in Canada, this would probably force a re-jig of student aid policy due to the 12-week rule).

It’s pretty clear this approach can work in fields like business (which is where it was piloted at SNHU and the University of Charleston); it’s also clear that it can be implemented in a relatively short period of time provided you’re prepared to pay a load of cash for faculty release so they can do the hard slog of program and course re-design. What’s not clear is whether this model actually works in a field other than business. Even SNHU and Charleston – both of whom are fairly evangelical about the 3-year degree – have yet to really see the model break into other areas of study.

There’s no reason it can’t be done, of course. We could go the route of the European Credit Transfer System and base credits on expected hours of work, not seat time. We could re-design whole programs around outcomes, and then put a whole lot more thought into course-sequencing so that students would continue to acquire competencies in a consistent fashion across the curriculum. This is all possible. But what it would really mean – in the arts and sciences, anyway – is ditching the whole smorgasboard curriculum we’ve had since the 1960s. The outcomes-based approach only really works if you keep relatively tight control of courses taken; too much student freedom messes up the careful laddering of competencies.

There’s a case to be made that this kind of change saves money and results in better outcomes. But even if it is the “right” thing to do, it’s not clear that either students or professors are interested. Bluntly, we prefer the inefficiency of the present system. Overcoming that preference is the biggest barrier to three-year degrees.

August 22

Compressing to Three

As we noted yesterday, there are four ways to go about getting university degrees from four years to three. One, cutting grad requirements from 120 to 90 credits, isn’t serious. A second, upping the use of prior learning assessment (or, in extremis, bringing back grade 13), is barely half-serious. That leaves curriculum compression and curriculum re-design.

Curriculum compression is the significantly easier path. No need to change anything other than the speed of students’ path through the system. By getting them to take more summer courses and six courses per term rather than five, they’ll get through that much quicker, saving everyone money.

That’s the theory, anyway. What about in practice? Well, to start with, it’s really not clear that students are interested in this concept. Students can already take six credits a term and take summer credits; they simply choose not to do so. In fact, they are far more likely to extend their programs for four years to five rather than shorten them to three.

There are some pretty basic reasons for this. Students tend to prefer work over school during the summer because they need money. Summer income accounts for something like 65% of all student income, and for a substantial portion of students, it’s their only income. A large number of others prefer to take reduced course loads in order to better balance part-time work with their studies: it’s a lot easier to get four A’s working fifteen hours a week than it is to get five, so students adjust their schedules accordingly. Getting these students to go from four to five classes – let alone six – requires money, and lots of it. They would need to be compensated for lost income, much of which would need to come as grants rather than loans in order to be effective.

How would faster completion save money, then? To some degree at least, student costs align with courses taken rather than time-on-campus; if the number of courses required to get to 120 credits isn’t reduced, it’s not clear how much costs would actually be reduced, other than by increasing class sizes. Come to think of it, this is actually the only way that compression can save cash. Kicking student load up from five courses to six while keeping professor contact hours constant would increase the ratio of outputs to inputs (i.e., “productivity”) by twenty percent. Assuming, that is, that your classrooms are big enough to handle the overflow.

Ceteris paribus, that might save a few hundred million in costs – but that money would almost certainly be consumed by the required increases in student aid. Is that really what the Minister had in mind?

August 21

Four Ways to Three

The Ontario Minister of Training, Colleges and Universities, Glen Murray, has a bee in his bonnet about three-year undergraduate degrees. Basically, he’s been told there’s some fiscal consolidation coming, and he thinks three-year degrees are the way that institutions can deal with the coming troubles without – allegedly – affecting quality.

There’s nothing inherently wrong with three-year degrees. All over Europe they are now standard (although in many countries, 80-90% of bachelor’s graduates go on to do a two-year Master’s degrees, so this might not be as felicitous an example as you’d think). And, of course, it’s not much more than a decade ago that three-year degrees were here in Ontario, too. It can be done.

The issue is – how?

It’s not simply a matter of rolling back time and going back to the 1990s. Back then, one of the rationales for having three year degrees was the existence of a fifth year of high-school. Ontario then – like Quebec today – was still on a continent-wide K-16 standard; the difference was just that the transition points within were slightly out of whack. What the Minister is proposing now is something different: K-15 vs. K-16. That really would put Ontario out of sync with the rest of the continent, with possibly some adverse consequences for student mobility in and out of the province.

There are basically four strategies for reducing undergraduate degrees to three years. The simplest – and most foolish – is simply to lop a year off the degree. Make 90 credits the standard instead of 120. It could be done overnight. I’m not sure what it would actually accomplish, but it could be done overnight.

The next simplest would be to punch up a lot of prior learning. Get more secondary students in Advanced Placement or International Baccalaureate courses in high school, and you can start handing out more course exemptions and move students through undergraduate education a lot faster. At the extreme, of course, you could always bring back grade 13. If cost-per-student-per-year is really the issue, getting first-year students out of research-intensive institutions and into secondary schools is about as cost-effective as you’re going to get.

That leaves two other strategies. The first is to allow – or maybe “encourage” is a better word – students to go through the existing system faster. Get them to take a sixth course each term, or more summer courses – anything to get to 120 faster. This is what might be called a “compression” system. The other is to actually re-design degrees from the ground up, mapping desired learning outcomes and working out how to get students to display desired competencies in a three-year period. We’ll look at these over the next couple of posts.

August 20

That International Education Report

The Federal Task Force on International Education reported last week. It was… how to put this? Very Canadian.

In essence, the report reads as though the goal of keeping all major stakeholders sweet trumped the goal of providing clear, bold thinking about Canada’s internationalization strategy. It’s worthy without challenging any conventional thinking. It puts forward an ambitious goal without spending much time working out the details of getting it done (the phrase “stakeholders should co-ordinate” does too much work in this paper). The words “social media” get waved around like a magic wand. The need to prioritize between markets was punted (recommendation 6 essentially says we should prioritize all foreign markets that everyone else already prioritizes).

Take, for instance, the issue of the goals of internationalization. There’s a fundamental tension between the idea of international education as an export product and that of international education as a talent magnet. The former implies that we are in it for the short- and medium-term financial benefits. Like Australia, we’d be selling seats in our very good universities to people from poorer countries so that we can subsidize our own domestic education system. If this is the goal, we can’t be terribly choosy about who comes here—if you’ve got money and a minimum of ability, we’ll take you. One thing it definitely does not imply is scholarships—if you’re in it for the money, subsidies actively miss the point.

The latter idea—internationalization for immigration—implies that money in the short-term is secondary. Basically, like the U.S. before about 2010, we really just want to cherry-pick the best and the brightest from abroad and entice them here with juicy research grants. In this strategy, scholarships make complete sense, but doubling the volume of students to somewhere in the neighbourhood of 450,000 per year does not; it’s quality, not quantity, that matters. Neither, in that case, should we be spending time chasing after high-school students (foreign high-school students who apply to Canadian universities are usually academically weaker than ones who come direct from their own country—their goal in coming here before the age of 18 is to improve their chances of getting admitted).

There’s nothing inherently wrong with either of these strategies, nor even anything wrong with pursuing both simultaneously. But they’re different, and they need to be resourced, pursued and evaluated separately as well, even if they both form part of a larger grand vision. A strong strategy document would have mapped this out carefully; what the task force presented instead was a jumble of worthy goals and promising-but-not-radical tactics. There’s nothing particularly wrong with the report, but there was an opportunity to do much more.

Page 18 of 40« First...10...1617181920...3040...Last »