It’s high time I started blogging again, so let’s start off with something that my colleagues in the library have been talking about at recent conferences — the link between the usage of library services and the final academic grades achieved by students.

As a bit of background to this, it’s probably worth mentioning that we’ve had an ongoing project (since 2006?) in the library looking at non and low-usage of library resources. That project has helped identify the long term trends in book borrowing, e-resource usage and library visits by the students at Huddersfield. Plus, we’ve used that information to help identify specific courses and cohorts of students who probably aren’t using the library as much as they should be, as well as when is the most effective time during a course to do refresher training.

Towards the back end of last year, we worked with the Student Records Team to build up a profile of library usage by the previous 2 years worth of graduates. For each graduate, we compared their final degree grade with their last 3 years of library usage data — specifically:

- Items loaned — how many things did they borrow from the library?
- MetaLib/AthensDA logins — how often did they access e-resources?
- Entry stats — how many times did they venture in to the library?

Now, I’ll be the first to admit that these are basic & crude measures…

- A student might borrow many items, but maybe he’s just working his way through our DVD collection for fun.
- A login to MetaLib doesn’t tell you what they looked at or how long they used our e-resources.
- Students might (and do) come into the library for purely social reasons.
- Using the library is just one part of the overall academic experience.

…but they are rough indicators, useful for a quick initial check to see if there is a correlation. Plus, we know from the non & low-usage project that there are still many students who (for many reasons) don’t use the library much.

So, let’s churn the data!

Here’s the average usage by the 3,400 or so undergraduate degree students who graduated with an honour in the 2007/8 academic year:

In terms of visits to the library, there’s no overall correlation — the average number of visits per student ranges from 109 to 120 — although we do seem some correlation at the level of individual courses. What does this tell us (if anything)? I’d say it’s evidence that the library is for everyone, regardless of their ability and academic prowess.

We do see a correlation with stock usage and e-resource usage. Those who achieved a first (1) on average borrowed twice as many items as those who got a third (3) and logged into MetaLib/AthensDA to access e-resources 3.5 times as much. The correlation is fairly linear across the grades, although there’s a noticable jump up in e-resource usage (when compared to stock borrowing) in those who gained a first.

Now the data for the 3,200 or students from the following academic year, 2008/9:

As before, no particular correlation with visits to the library, but a noticeable correlation with stock & e-resource usage. Again we see that jump in e-resource usage for those who got the highest grade.

Note too that the average usage has increased. We’ve not changed the way we measure logins or item circulation, so this is a real year-on-year growth. (Side note: as we make the move from MetaLib to Summon, the concept of an “e-resource login” will change dramatically, so we won’t be able to accurately compare year-on-year in future)

Finally, here’s both years of graduates usage combined onto a single graph:

I’m curious about that jump in e-resource usage. Does it mean, to gain the best marks, students need to be looking online for the best journal articles, rather than relying on the printed page? If that is the case, will Summon have a measurably positive impact on improving grades (it certainly makes it a lot easier to find relevant articles quickly)?

Going forward, we’ve still got a lot of work to do drilling down into the data — analysing it by individual courses, looking deeper into the books that were borrowed and the e-resources that were accessed, etc. We’re also need to prove that all this has a stastical relevance. Not only that, but how can we use that knowledge and insight to improve the services which the library offers — it’d be foolish to say “borrow more books and you’ll get better grades”, but maybe we can continue to help guide students to the most relevant materials for their students.

It’s all exciting stuff and, believe me, the University of Huddersfield Library is a great environment to work in… I just wish there were more hours in the day!

This is really amazing data!

I wish we had such data here. You must have spent a lot of time getting the usage data.

“it’d be foolish to say “borrow more books and you’ll get better grades” – Definitely, I’m sure you know correlation does not imply causation, chances are students who are the type that are inclined to get first class, naturally like to use library resources more…

Looking forward to learn more about this fascinating project whether in a series of blog posts or a paper.

Thanks for sharing!

I would be very interested to see what happens when you analyze this by course. My first thought was that I had two majors (US university system), and for one of them I had a lot more library usage, and my GPA in that major was significantly better…

…but that was my humanities major, which both required much more library use due to the type of assignments, and simply had much easier classes. I can only think of one assignment in my entire math major which required library use — most of what I needed for math was textbooks, pencil, and paper — but I was also asked to do, in general, harder things, and held to a harder (and more objective) standard in grading. And it seems, over here at least, to be a truism that tech classes have lower grades than humanities classes.

So I wonder if you are seeing a real phenomenon here, or just two correlations; that between major and library use, and that between major and grades.

I’m also interested if you see any patron privacy concerns (or student-privacy concerns for the registrar’s office) in using data in this way. In the US there are some strong restrictions on who gets to see student grade data, but I don’t know how it works where you are.

Don’t get me wrong — I think this is fascinating and I started following your blog because your experimental use of circ data in your catalog intrigues me (and I wish more people were messing around to see what they can build!). But it is in my nature to be skeptical. (Or sceptical, if you prefer.)

Andromeda — the principle legislation in the UK is the Data Protection Act 1998. I’m not overly familiar with US legislation, but I get the feeling there is wider scope for doing this kind of analysis in the UK. For all intents and purposes, we’re working with aggregated data sets: rather than examining what named individuals did, we’re instead looking at what groups of students with a common attribute did (e.g. all 1st year students, all students in a school/faculty, or all students on a specific course).

The usage by course varies widely (as you say, humanties library usage is much higher than the tech courses), but the grade correlation seems to be fairly universal. As you suggest, we need to do more work on finding the underlying causes of the correlation(s).

One thing I forgot to mention in the blog post is that, in general, the correlation seems to be present in the 1st year of study — in other words, the students who will eventually achieve the highest grades are already using library resources more than those who will go on to get lower grades. So, one area of exploration might be to survey current students on their library usage prior to joining the university — are the schools and colleges that have information literacy embedded in the curriculum the ones that put their students on the path to achieving better grades? How big a factor is prior use of public libraries?

Aaron — several of my colleagues in the library have (or are in the process of) writing papers, so I’ll try and collate the details.

That first-year correlation is really interesting. I know that the UK system is much more final-exam oriented than the US but I don’t know the details; do you have any way of looking at achievement levels before the final degree (and thus, e.g., see if first-year students are already doing a lot better if they’re heavy library users, or see if there’s a gap that widens, perhaps due to better study habits, over the course of the degree)? And yeah, the ability to extrapolate backward through K-12 schooling would be awesome too.

Good to see that there’s some possibility of examining correlations while holding course of study constant, too.

Hello Dave

As you know we at Greenwich have also been doing a bit of work in the same area.

Our correlation is almost exactly the same as yours; the higher the final grade the more library usage. However when we drilled down to just school level there were marked differences between them: in one school the more items borrowed the worse the grade. What does that say? Maybe these students are not understanding the course and are relying too heavily on text books. Or maybe we have rubbish print material for that school. Either way it has been an interesting project and something we are looking to carry on for the foreseeable future.

Hi Iain

Sorry — I tweeted about Greenwich but totally forgot to include you in the blog post!

We’ve seen a similar strong negative correlation with visits to the library on a couple of courses. Again, it’ll be interesting to get to the bottom of why that is — “want to get a bad grade? Come to the library!” ;-D

At a City of Bristol College we did a similar analysis of A level students their final grades and the number of times they logged onto the VLE or online teaching environment.

We found a similar correlation in that students obtaining better grades had a higher average number of login’s to the system. There were of course exceptions (for example a number of staff didnt use the VLE so their students didnt tend to log on much).

What I thought was useful was that you could use the stats to plan interventions as use in the first 2 months showed a similar pattern to use for the full 2 years(and you could use yours in the same way).

It might be possible to tell a Teacher – We are 3 months into the course, these students have a poor number of login’s and book issues, this implies they may achieve poor final grades. Then you can speak to them and explain the importance of reading around the subject and keeping up with the course materials.

In fact it would be a very interesting study to see what affect these interventions had (basically run a number of groups where tutor has targeted meeting with low login students and have control groups where there isnt an intervention) and see if at the end of the course the intervention group had compared to the control groups A) fewer log login students B) fewer lower grades. If A and B happen then interventions could prevent some students from under achieving, if A and not B then less able student tend to take out less books but making them take out more doesnt help, if not A and not B then your intervention was ignored.

Ian, Dave,

I would be very interested to know which schools appeared to have a negative correlation