Measuring the effects of student loans is brutally difficult.  It sounds simple, but it’s not.

Take a recent article called “Gender, Debt, and Dropping out of College“, published in Gender and Society, which made a small wave in access-conscious circles a couple of weeks ago.  Using data form the 1997 US National Longitudinal Survey of Youth, this article made two claims: first, that debt was positively correlated to completion up until a certain level of debt, after which the relationship reverses itself somewhat (though the relationship remains at all times positive); and, second, that this effect was greater among men than women – that is, at any given level of debt, the positive effect on graduation was more pronounced among women than men.

(Unfortunately, in the online commentary, the last point tended to be summarized as, “debt has more of an adverse effect on men than women”, which isn’t the same thing at all).

Now, debt is not an exogenous variable.   It doesn’t come out of nowhere.  It implies financial need, which, in turn, is a function of both socio-economic background and academic achievement.  It also implies a preference for borrowing over working.   Persistence is also a mutli-dimensional issue.  As generations of researchers have discovered, since Vince Tinto started working in this field nearly 40 years ago, there are a whole host of inter-correlated factors that affect persistence, of which finances are just one. So linking the two causally is a tall order.

One of the best papers ever on this subject was written a few years ago by University of Ottawa professor, Kathleen Day.  Using Canada’s Youth in Transition Survey, she found a small but positive relationship between student aid and perseverance in studies (though when “aid” was disaggregated into loans, grants, and scholarships, the effect was no longer detectable).   She then re-estimated her models to account for the possibility that aid and debt were both related to another unobserved variable, and found that the relationship between aid and persistence turned negative.  (That’s not to say aid is ineffective, but rather that the available data doesn’t permit a conclusive answer.)

Back to the Gender and Society paper, which has a variety of flag-raising methodological limitations.  There are no controls for student labour market participation, for a start.    Family income is treated as a trichotomous variable (high, medium, low) rather than a continuous one.  The paper also seems to not account for credits accumulated, or number of years enrolled, which is very odd.  Most seriously, it doesn’t account for the possible unobserved correlated variables.

Given this, I’d take the article’s conclusions with a truckload of salt.  It’s interesting, but far from conclusive.  We’re still a long way from understanding the effects of debt.