Nowadays, every Canadian university and college wants to be seen as caring about and contributing to their community.  “WE CARE ABOUT COMMUNITY”, they say.  But if someone asks “how so?  How do we know this, and how do we know if you are effective in your community efforts?”, the answer is usually a lot of handwaving.  “We do stuff!” comes the reply, while gesturing towards descriptions of a bunch of things they do.  

The idea that intentions rather than results are what counts is very Canadian.  God forbid anyone be held accountable for something as trivial as results.

Let me show you a couple of ways other places around the world are handling it.  Exhibit A comes out of some work done by a group of mainly Australian and UK universities over the past few years through a series of “Global University Engagement Summits.” The first was held at Monash University in 2017; the second was held at the University of Manchester in 2019.  The idea coming out of both summits was that a) engagement was critical to the University enterprise and b) it was possible to measure, track and compare it.  And so some of the universities most identified with this enterprise – Monash, Chicago and King’s College London, decided to engage the Australian consultants Nous Group (same folks who helped the University of Alberta with their restructuring work last year) to put together a paper on how they might do it.

What they came up with were eight metrics, which in theory look at the right kinds of things.  But the actual metrics?  Well, let’s be polite and say they illustrate how difficult this stuff is to measure. The table below shows the proposed indication:

Indicator	Measurement	Credibility
“University commitment to engagement”	Institutional documents, not necessarily public, outlining institutional & senior admin responsibilities for engagement.	Almost zero
Community opinion of the university	Survey of university partners	OK if you can get credibly equivalent samples, otherwise low
Student Access	Participation in bridging programs	Very low.
Volunteering	% of students and staff volunteering in programs run by the university multiplied by hours	I can’t imagine this capturing more than a fraction of the true time commitment.
Research reach outside academic journal	Using altmetrics, total mentions in grey lit, media, policy papers etc as a percentage of total metrics tracked	Interesting but untested.
Community Engaged Learning	% of students participating in a credit-earning class which has a practical community engagement element	In theory interesting, suspect a lot of difficulty in harmonizing definitions here
Socially-responsible purchasing	% of university budget spent on procurement “linked to social benefit”?	I have no idea what this means or what it is measuring.
Carbon footprint	Emissions per student	Credible, though usual difficulties for cold-climate unis

But of course, measuring things via internationally-comparable indicators isn’t the only way to go.  Let me also introduce you to a European initiative known as TEFCE, a somewhat clunky acronym for Towards a European Framework for Community Engagement in Higher Education.  This is less an attempt to aim for comparability than it is to get institution to reflect in a meaningful and evidence-based with their own activities.

TEFCE’s main output is a “toolbox” which helps institutions conduct a deep and structured self-examination of their engagement activities.   It’s a bit long to describe here, but basically it shows institutions how to measure levels of engagement based on a standard rubric, how to create a kind of “heatmap” of their engagement, and how to analyze areas of strength and weakness, while at the same time creating space for dialogue within the institution about how to improve.  Here’s a quick schematic diagram of how it works:

As a self-improvement tool, TEFCE’s toolbox seems useful – almost certainly more useful than what the Nous Group has developed, at least for those institutions that are prepared to take the time and effort to really follow this path (as a process it is neither simple nor easy). I suppose the difference is whether you think universities have the capacity to improve engagement solely on their own initiative, or whether they need to be prodded through external competition through rankings. 

Or, in the Canadian case, whether institutions have any interest at all in going beyond mere handwaving.  Here’s hoping some day they do.