Last Friday, I presented a six-minute lightning talk at the Library Assessments Workshop hosted by the Colorado Alliance of Research Libraries. Much to my supervisor’s chagrin, I was frantically putting together my slides and practicing my talk at the end of the day on Thursday. Some of the disjointed brainstorming notes looked like this:
- Don’t confuse satisfaction with learning.
- Self-efficacy (confidence) is important, but it’s not the same thing as applying skills.
- “I feel confident using the library catalog” doesn’t mean you can use it.
- What does this mean for assessment?
- Watch the language that you use in your student evaluations and reflections.
- “Helpful.” What does it mean that something is helpful? Or useful? That you’re comfortable working with the librarian?
I think I was able to pull together something coherent, but you can judge for yourself. My finished talk is available here (just the slides) and here (a five-minute recording). The article that I refer to, “Best methods for evaluating educational impact” by Schilling and Applegate (2012), is available here.
The day began with Megan Oakleaf joining us virtually from her office at Syracuse University to deliver a keynote address about the state of assessment in higher education today and the academic library’s role in current assessment trends. I admire Oakleaf’s contributions to academic librarianship and her leadership in assessment, but I disagreed with most of her presentation. It seems, perhaps, that I misunderstood her presentation as advocating for certain actions, e.g., collecting individual student-level data to affirm the library’s value in retention and persistence. According to her responses to me on Twitter (all of which she has deleted), she was simply the “bearer of news” (as opposed to an advocate for surveillance state academic libraries).
I think she made valid points about the role of accreditation–that there are many flaws with the accreditation process, it’s confusing, it varies a lot, and the library’s role in accreditation is usually focused on our spaces and collections, rather than on our impact on students. Accreditors expect higher education institutions to commit to continuous improvement, but don’t say how to do this (similar to how the ACRL Framework expects academic libraries to use the frames to guide information literacy outcomes, but doesn’t tell library instructors how to do it).
It’s hard to get any population to police itself, and faculty are not the most compliant animals.
Megan Oakleaf talking about accreditation in her Library Assessments Workshop keynote
Oakleaf differentiated the concepts of student learning and student success. I also agree that student learning and student success are not the same thing, and the metrics we use for success (persisting from year to year, graduating within a certain amounting of time) do not necessarily measure gains in student learning. In Oakleaf’s words, “Student success surrogates do not equal learning.” Amen.
In the remarks that followed, Oakleaf emphasized that libraries need to situate their value within their individual institutional contexts. “Libraries have stuff, know stuff, and do stuff,” she said. The key is to connect all of that “stuff” to your bigger institutional picture, and to show that your library makes an impact in the lives of your stakeholders (including students, faculty, alumni, community, and so on). I’m definitely on board with all of this.
The part where she lost me (and when I had to take off my blazer because I was sweating so hard) was when she said,
We won’t know what difference the library makes until we collect data on individual library users.
Megan Oakleaf talking about the role of analytics in library assessment in her Library Assessments Workshop keynote
I am one of those people who has, in Oakleaf’s words, a “visceral reaction” to such an assertion. I believe in the Library Bill of Rights and the ALA Code of Ethics, like really believe in them, so yeah, I don’t believe in (or value) tracking individual users beyond the extent necessary to provide our sources (e.g., we keep track of a book that’s checked out while it’s gone, but once it’s back, we delete any record that you ever had it, and we don’t tell anyone else what you’ve been reading, either).
I fundamentally do not see the value in correlating an individual’s library use with their ability to persist in college. I think there are better, more meaningful ways to tell the story of our value and the impact we have on our stakeholders. I loved the example that Oakleaf included about the “Critical Incident” studies by Ross Todd, where students in K-12 libraries were asked about a time that the library helped them. Students were asked, “What was the help you got from the library, and what were you able to do because of it?” Asking more questions like this, and listening to our stakeholders, is far more valuable to me than correlational data gained from surveilling and tracking students.
I get the sense, and I could be wrong, that all of this is theoretical to Oakleaf, who does not work directly with undergraduate student populations and has not done so in over ten years. Her last position as an instruction & reference librarian ended in 2006 when she became faculty at the School of Information Studies. Maybe it feels abstract to advocate for the privacy rights of students when you don’t work with them everyday. At this point, it would be fair to allow Oakleaf to explain herself a bit more, and I would be glad to include her own words here. As I mentioned above, she engaged with me enthusiastically on Twitter and responded to nearly all of the tweets I wrote during her presentation–but I should have taken screenshots of her responses because she has has since deleted everything she said to me. This is unfortunate because I wanted to include her remarks here as a way of balancing my interpretation of her keynote.
One of the things I like about having this blog is that I can be wrong, publicly wrong, and I can come back later and read what I thought and see how wrong I was. Perhaps that will be the case here, and I’ll come back in a year or two or five and laugh at how silly I was for resisting the idea of using student data to prove library impact. For right now, though, I will hold fast to what I believe, and you can thank me in advance for freeing up a seat at Oakleaf’s panel at ACRL2017 about the “responsible use of library data”–instead, I’ll be at Eamon Tewell’s session, “Asking, Listening, Observing: Learning about Student Research through Ethnography.” Sounds like my kinda jam.
Zoe Fisher 