A couple of weeks ago, the World Bank published a very interesting little paper which received little attention.  What Works to Reduce Inequalities in Higher Education?  A Systematic Review of the (Quasi-) Experimental Literature of Outreach and Financial Aid, by Koen Geven and Estelle Herbaut, needs to be read by everyone with an interest in expanding access to higher education. 

While there have been many papers which have used meta-analysis techniques to look at financial aid programs, this paper extends those analyses in three ways.  First of all, it looks not just at financial aid programs, but also at various types of programs which involve information, guidance, or extra tutoring.  Second, it looks not at “what expands access” but specifically what expands access for disadvantaged students – a crucial distinction some studies miss.  Third, and probably more importantly, it restricts itself to studies of an experimental or quasi-experimental nature. All of this together means the paper covers a lot of important ground in a relatively few pages.

Now I do urge everyone to read the whole thing (and you guys always do that with the documents I link to, right?), but very briefly, here’s what the report found:

• Interventions which are information-only – that is, involve providing students with extra information about the benefits of attending post-secondary education – mostly show effects which are close to zero (one exception was a study conducted in China which may suggest that the effects of such programs may be different in countries with lower GDP/capita).  Note though that the study does not control for the student’s age at the time of the intervention – it is possible that these interventions are happening in the final year or two of school when preferences have already been formed and efforts in earlier years might be more effective.

• In contrast, interventions which combine information with personalized assistance or guidance have quite a strong impact.  That said, these are generally more expensive to provide (because they take more time and effort), which is why they are less common.  But I think this underlines a point I made in a paper with co-authors Dominic Orr, Cezar Haj, Irina Geanta and Graeme Atherton a couple of years ago: guidance is a (literally) a class issue, a way of bridging gaps in cultural and academic capital. Even small amounts of assistance can help; studies which test only “nudges” (e.g. reminding students via text to take certain steps towards enrolment, such as filing for financial aid) have been found those nudges to be effective to some degree.

• Interventions which involve extra academic tutoring, interestingly, appear to have very little effect on access.  The authors suspect this has to do with high drop-out rates in such programs.

• Interventions which use need-based grants worked to increase enrolments, but the size of the grant needed to be significant in order to have an effect.  Intriguingly, the report finds examples in Europe which confirm American research on this subject that the enrolment effect is about 3 percentage points for every $1,000 US in aid. 

• Merit-based grants were found to actually have slight negative effects on enrolments unless there was also a need criteria attached.

• Loans appear to have positively massive positive effects on enrolment, though this finding is based on relatively few findings (and in fairness because of the way loan programs are run, quasi-experimental evidence tends to be hard to come by); conversely, tax credits are not found to have any effect on enrolment (HEY NEW BRUNSWICK, ARE YOU LISTENING?).  

• Perhaps unsurprisingly, programs which combined outreach with financial aid were usually found to be more effective than programs which only did one or the other.

Now, a couple of caveats about this work.  Because it is looking at experimental evidence it considers the effects of what might be called “specific changes that move the needle”.  By definition, it can’t tell you whether one level of tuition or student aid is better than another; it just tells you the effects of changes to the status quo (regardless of where that status quo may be).  Also, while the scope of the coverage is impressively global – it includes studies from four continents – the vast majority of them are from the United States (this is partly a reflection of the regional distribution of research interests but mostly because genuine quasi-experimental conditions are rare outside the US).  So, these results may not hold everywhere; that said, my guess is they would mostly hold in Canada given the similarity between the two countries.

All in all, this is a very good study. If you have been paying attention to access studies for the past decade or so, there is nothing in it that will surprise you: outreach matters, grants matter, cheap interventions tend not to work, but at least in cases where credit constraints exist, loans can be very effective as a way of increasing attendance.  But Herbaut and Geven have done everyone a service by putting all the highest-quality evidence together in a single study.  It deserves wide readership.