I was reading the Ontario Confederation of University Faculty Association (OCUFA)’s position statement on a new funding formula for the province. Two things caught my eye. One, they want money to make sure Ontario universities can do world-class research and teaching; and two, they demand strict opposition to any kind of performance-based funding formula (PBF). Put differently: OCUFA wants great teaching and research to be funded, but are adamantly opposed to rewarding anyone for actually doing it.
Except that’s slightly uncharitable. OCUFA’s larger point seems to be that performance-based funding formulae (also known as output-based funding) “don’t actually achieve their goals”, and point to work done by University of Wisconsin professor Nicholas Hillman and Florida State’s David Tandberg on the topic. From a government-spending efficacy point of view, this objection is fair enough, but it’s a bit peculiar from an institutional or faculty standpoint; the Hillman/Tandberg evidence doesn’t indicate that institutions were actually harmed in any way by the introduction of said arrangements, so what’s the problem?
Anyways, last week HCM associates in Washington put out a paper taking a contrary view to Hillman/Tandberg, so we now have some live controversy to talk about. Tomorrow, I’ll examine the Hillman/Tandberg and HCM evidence to evaluate the claims of each, but today I want to go through what output-based funding mechanisms can actually look like, and in the process show how difficult it is for meta-analyses – such as Hillman’s and HCM’s – to calculate potential impact.
At one level, PBF is simple: you pay for what comes out of universities rather than what goes in. So: don’t pay for bums in seats, pay for graduates; don’t pay based on research grants earned, pay based on articles published in top journals, etc. But the way these get paid-out can vary widely, so their impacts are not all the same.
Take graduation numbers, which happens to be the simplest and most common indicator used in PBFs. A government could literally pay a certain amount per graduate – or maybe “weighted graduate” to take account of different costs by field of study. It could pay each institution based on its share of total graduates or weighted graduates. It could give each institution a target number of graduates (based on size and current degree of selectivity, perhaps) and pay out 100% of a value if it hits the target, and 0% if it does not. Or, it could set a target and then pay a pro-rated amount based on how well the institution did vis-a-vis the target. And so on, and so forth.
Each of these methods of paying out PBF money plainly has different distributional consequences. However, if you’re trying to work out whether output-based funding actually affects institutional outcomes, then the distributional consequence is only of secondary importance. What matters more is how different the distributional outcomes are from whatever distribution existed in the previous funding formula.
So, say the province Saskatchewan moves from its current mix of historical grant and formula grant to a fully PBF system, where 100% of the funding is based on the number of (field-weighted) graduates produced. Currently, the University of Saskatchewan gets around three times as much in total operating grants as the University of Regina. If USask also produced three times as many (field-weighted) graduates as URegina, even the shift to a 100% PBF model wouldn’t change anything in terms of distribution, and hence would have limited consequences in terms of policy and (presumably) outputs.
In effect, the real question is: how much funding, which was formerly “locked-in”, becomes “at-risk” during the shift to PBF? If the answer is zero, then it’s not much of a surprise that institutional behaviour doesn’t change either.
Tomorrow: a look at the duelling America research papers on PBF.
Tweet this post
While I know that you have a bit of a constraint in the length of the posts, it’s a bit misleading to suggest that the “big” issue is distributional or even “locked-in” vs. “at risk”. That might be an issue between institutions, but as a government funding formula, there are two much bigger issues.
First and foremost, universities can cream-pick good students and good students only. That seems like a difficult thing to do, but it isn’t. If you get funding for those who graduate, and none or less for those who don’t, you want to make sure that the people you let in are actually capable of making it through. It happens in other services all the time with PfP — if you want the numbers up, don’t take anyone with a cognitive disability, or lower economic background, or divorced parents, or a whole host of other sociological factors that you can exclude as a group and thus increase the likelihood of graduation. It’s one of the reasons some of the more elite programs have much higher completion rates.
Second, and more dangerous, is the desire to “promote for performance pay” i.e. even if they can’t spell or even read, a PpF grade school still promotes the kid into high school who then promotes them and graduates them — can’t have a funding cut and not get reimbursed for the work done to that point. So standards get lowered — “Everybody graduates! Yay!”.
There are, of course, solutions to that problem including partial lock-ins or differentiated performance pays i.e. if you take a regular client and get them to completion, it’s 100%; take someone from a disadvantaged group and get them to the same point, you get 120% (which sounds like a reward, but is also likely it cost you more in staff time to help that person get to completion). Or move to a “partial” system of reward.
There are also other types of perversions with the “management by outcomes approach”. In health care, there is an example that is used regularly with VA stats in the U.S. Service standard was, say, 90 days to see medical specialist once you’ve been initially assessed. The perversions were then two-fold — don’t complete the assessment until “all the info” has been provided (like asking for the maternal grandmother’s maiden name — not needed, but sorry, patient hasn’t completed their part, not the hospital’s problem) and then later, if the person doesn’t see someone for 90 days and slips to 91, at that point, there is no reason to ever rush to help them…120 days, 200 days, 300 days, makes no difference because the funding was already “lost”. Better to spend time helpign those under 90days where you can still get funding. Happens in triage rooms too where less serious patients were often helped ahead of those with more serious injuries who had already gone past the “performance pay” clock, because helping the needy patient would cost more lapses in p-pay.
I’m sure there are other perversions in the education sector at the university level, but rarely is a “one size performance funding” model going to work well for anyone.
Poly
Hi Alex,
Your “locked in” vs “at risk” argument seems to suggest the impact is entirely through income effects. But there are relative price effects too. If I put a 30% HST rate on food but refund the tax revenue, people will have as much income as before. But they will consume less food and more other things.
Same here: relative prices change, so Unis will produce more of things that had prices go up; less of things that had prices go down.
Kevin – fair point. That wasn’t the best choice of words on my part. What I actually was trying to convey was that sometimes the new set of prices don;t in practice look a lot different from the old set of prices (because they’re based on indicators which at heart are covariate) So what you really need to look at is change in prices not “what is the new price”? Although the most state of the art research is actually one step behind this – it’s still at the “are there new prices?” stage.
Paul has given some good examples, though I think they’re examples of Goodhardt’s law: “When a measure becomes a target, it ceases to be a good measure.” Most ways to meet a target are perverse, which requires more measures, which eventually produces research assessment exercise Hell.
Secondly, this really seems designed to allow governments to micro-manage universities, by telling them what disciplines they ought to teach, for instance, instead of simply funding them on some simple formula and letting them decide based on established areas of expertise, or something else of academic relevance.
1. PBF is about as simple a formula as you can get (I mean, I suppose you can make it complicated, but in principle it is pretty straight-forward).
2. You don’t think govt has a right to say: “here is what we would actually like you to accomplish with our money”? You really think the public sees universities as existing only for the purpose of “academic relevance”?
I didn’t say it’s too complicated in principle: I said it’s too complicated in result, since the simple rules produce perverse results, requiring further rules, and so forth, all the way down to Hell.
And no, I don’t trust governments to say what should be done by universities, any more than I trust them to tell courts what sorts of judgements to make. Universities are not branches of the civil service; moreover, the sort of expertise they cultivate takes decades, if not generations to build. Making their work subject to the whims of ignoramuses who only care about meeting social and economic needs is precisely how the Soviet system became an intellectual wasteland.
Ideally — it won’t happen, hence “ideally” — governments would endow universities, or at least individual chairs and programs, so that universities could get on with pursuing knowledge in perpetuity and free of political interference.
So you’re in the “take their money but ignore them” camp?
I’m in the “universities are a public good best served by just being universities” camp. Are you in “the courts should do what they’re told because they’re government-funded” camp?
This idea that “universities” could “just be universities” is just odd, though. Universities change enormously over time. Research universities began in Germany *specifically* “to strengthen the states” (and indeed, until quite recently professors were civil servants in most of Europe). In Canada, the idea that original research was essential to the identity of a university would have sounded quite ludicrous until fairly recently 60s or even maybe 70s. Changes happen over time and they are negotiated with the state (which still funds universities to the tune of about 60-65% in Canada once you thrown in the research dollars).
I mean, if you want universities to forego public money that’s one thing; otherwise, seems to me there has to be some give and take.
And I can assure that if “meeting labour force needs” were not something universities already did tolerably well, they would get far less funding than they currently do.
Strangely, there’s no “reply” any more on this thread. Perhaps it’s an issue with the fact that I’m using chrome. By the way, I was too glib with my last response, so it’s decent of you to respond in the way you have.
Yes, universities were, in many cases, created or strengthened to help states. When they’re really closely managed, however — as in tyrannies of any kind, but also in other circumstances — they cease to be very helpful. Even the Prussian state university system rather shot itself in the foot by driving Burckhardt and Nietzsche to Basel as refugees. It certainly laid down the roots of the Nazi campus, which drove Einstein out. More recently, Peter Higgs claims that he was nearly fired by the University of Edinburgh under the Research Assessment Exercise. The closely-managed European universities, everyone seems to agree, have fallen badly behind their freer North American counterparts. I recall C. P. Snow writing that the Oxbridge colleges hit their intellectual nadir when they functioned most obviously as training institutes for the Church of England.
Perhaps this is all to say that the ideal university has never existed, but it doesn’t have to in order to serve as an ideal: we should do those things which move us nearer to it, and avoid those which detract from it. Another way to put all this: the tradition is of academic independence, or at least the traditional value is of academic independence, whatever the empirical experience has been. This is the tradition we should attempt to follow and strengthen.
And as you say, universities prepare workers for the economy reasonably well. Somewhat paradoxically they this best when they don’t “prepare workers for the economy” but try to educate people, according to the lights of their own learning and tradition. Engineering schools function best when they don’t become engineer schools, and law schools like to protest that they aren’t lawyer schools. Similarly, someone with a liberal arts background is often very well prepared for all sorts of different work, as well as to take her or his place in public life. If we tried to reduce the humanities to (say) basic literacy and critical thinking, taught through remedial writing classes and solving puzzles, we would do incalculable damage to our economy, state and society.
In research, while a good deal of funding is directed to specific outcomes — the atom bomb, man on the moon — often the most valuable outcomes aren’t the kind that can be directly measured and certainly weren’t foreseen in advance. Simply funding engineers or nuclear physicists because they were interested in engineering and nuclear physics would have produced much the same spin-offs, possibly at less cost. Goals from above, moreover, detract from everything else a university might do. There was a recent article in The Economist about how British universities are trying to build their Chinese programs. One has to dig into it a bit, to find that they actually have to re-build their Chinese program. Oxford’s was gradually shut down in favour of other things — German, especially — in the mid-twentieth century. Had they simply kept it running, they would have a stronger program than can now be built from scratch, and less reliant on the Confucius Institute. Unless each “strategic goal” is linked to new funding, there’s an opportunity cost to pursuing it.
And there’s always a cost to the paperwork in attempting to achieve goals. I recall reading a study to the effect that Australia alone has wasted 400 man-years — and man-years of extremely capable people — in the form of scientists applying for funding they didn’t receive. How many diseases could be cured in that time? How much more could we appreciate the mysteries of our lives and the natural world? Why do we so despise the labours of our finest minds?
Yours,
SKL
P.S. You may be interested in the two articles I refer to above, so here are their links:
http://www.economist.com/news/britain/21617026-britains-universities-are-finally-growing-interested-modern-china-re-orientation
http://www.nature.com/nature/journal/v495/n7441/full/495314d.html