Digital Reserves Assessment, Spring 2021

Since more than half the Swarthmore student studied remotely this academic year due to the COVID-19 pandemic, the libraries needed to shift very quickly from providing print Course Reserves to providing digital copies of all required course materials. In the Spring of 2021, the Assessment Committee began to gather data in order to learn the answer to the following question:

What are the benefits of digital reserves to faculty and students and what are the costs to the libraries?

METHODS:

  • Faculty: we personally invited faculty who are known to make reserves requests to provide feedback via a questionnaire
  • Students: we held one focus group via zoom and used results of a campus-wide student survey on a pilot textbook affordability program (TAP). The libraries were able to add four questions to the survey to ask whether course readings on reserve at the libraries impacted student decisions to purchase textbooks
  • Library Staff: we gathered feedback first via department meetings of relevant staff and then held one, larger, inter-departmental meeting for further discussion

WHAT WE LEARNED

Faculty: While some faculty expressed dissatisfaction with particular digital platforms, they were appreciative of  having digital copies of required course texts available from the libraries during the pandemic and  expressed a desire that digital options remain available after all students are able to return to campus. One professor noted “I think students consult the material when it’s at their fingertips, so digital is a good idea.” and another wrote “Digital Reserves are accessible [meaning available] and helpful. Please keep them if it’s at all feasible.”

Students also appreciated access to digital copies of reserves materials, however, a majority of students used Textbook Affordability Program (TAP) Funds to purchase most of their required texts from the Campus and Community Store. Students in the focus group expressed an appreciation for choices between their purchased, print textbooks and the digital options the library provided. Although they purchased print texts, they made use of the library digital copies for pulling quotes and quick searches.

The results of the TAP Fund Survey indicated the program is successful, with students using the funds to purchase a majority of their course materials. While using course reserves at the libraries was the 2nd most frequent avenue to required reading, survey results demonstrate that the libraries may revisit their policy of providing library copies of every required text in light of the continuing availability of TAP Funds.

The survey also provided interesting data on reading preferences with a majority of students indicating a preference for reading in print rather than digitally. The results indicate that students choose digital copies when they are far less expensive, or when there are no print options available.​​​​

Library Staff worked heroically to provide digital access to course materials during the pandemic, including developing workflows to make in-house scans of needed materials that were not available digitally via any other source. While staff understand the value and benefit of digital copies for faculty and students, they want to provide them responsibly, considering accessibility (in the sense of accommodation), copyright, and the storage and management of the digital files we create.

WHAT WE DECIDED

The libraries will:

  • Use a hybrid approach to acquire print plus digital copies wherever possible from Ebook vendors for Course Reserves
  • Prioritize acquiring digital copies for Reserves, and in light of the TAP Fund Survey results, re-evaluate our policy to provide at least one print copy of every required text
  • Continue to explore responsible and robust options for Controlled Digital Lending (CDL) as options are developed 

Streaming Film Assessment

This year, the libraries are doing an assessment of our film collections in order to better understand how changes in the film industry and technology, particularly streaming options, are impacting the libraries’ materials budget.

We’re investigating everything we can think of:

  • Purchase requests
  • Statistics from our licensed streaming packages
  • Statistics for video placed on reserve and checkouts
  • How many “ripping” requests we receive (to make DVDs available for streaming) and how much time this requires
  • Film related workflows from request to delivery
  • What supports and services for providing film might be duplicated or supported by our campus partners, academic technologists in ITS and the Language Resource Center.

So far, we’ve:

  • Gathered statistics on:
    • Our streaming packages
    • Films placed on Reserve
    • Netflix use
  • Spoken to colleagues in the Language Resource Center to learn about all the ways they support the use of film in the curriculum
  • Devoted a student Library Advisory Board (LabX) meeting to get feedback from students about their film use in order to learn:
    • How often they use the collections for study breaks and entertainment
      • whether they consider checking out DVDs
      • what the libraries might do to make watching DVDs more convenient
    • How they view films required for class:
      • what happens if their schedule prevents them from viewing a film outside of class at a time the professor arranges?
      • how they watch a required film that may be available only through a platform the libraries are unable to support, like Amazon Prime or Hulu?

Our next step will be to send a short survey to about twenty professors who are known to use a lot of film in their courses in order to learn:

  • About how many films they may require in a film-heavy course
  • Whether their need for film is growing
    • Are standard courses taught from year to year impacted?
    • Does film play a larger role in newer courses?
    • Does the proliferation of available content suggest growing need?
  • What drives their preference for format?
    • Availability?
    • Accommodation (captioning, multi-language subtitles)?
    • Copyright?
    • Big screen vs laptop?
    • Picture and sound quality?
    • Convenience of delivery?
  • Are provisions made for students to view film outside of class time?
  • What if a film they want to include is available only on a platform the library cannot support (such as Amazon Prime or Hulu?) What accommodations do they make for students?
  • How might the libraries best support their need for film?
    • How might we improve the request process?
    • Should we investigate the feasibility of providing students with Amazon Prime, Hulu or other streaming platform accounts?
    • What other ideas and opportunities could there be?

We hope our assessment will inform:

  • Whether the evidence we gather supports advocating for more dollars in the budget, or guides difficult choices in order to balance the budget and curricular need
  • How we might improve our workflows from purchase request to film delivery
  • Hidden opportunities that may improve our collections and services in the future

If you have ideas of other avenues to explore, please let anyone on the AUX Committee know! (Assessment and User Experience)

Alison Masterpasqua, Amy McColl, Peggy Seiden, Barb Weir, Mary Huissen, Pam Harris, Pat O’Donnell – with much support for this project from Jessica Brangiel and Nabil Kashyap

Dog Days of Summer

It’s a Friday afternoon in the midst of a punishing heatwave in Swarthmore during the dog days of summer…

Photo by Paulette Wooten on Unsplash

…we’re just past the midpoint of the calendar year and a new academic year is on the horizon. What a good time to take stock of some of the goals we have achieved so far and to think about new ones for the upcoming year.

Since starting in this position, I’m proudest of these things:

  • IRB approval for usability testing
  • Approval to hire students at the highest pay grade who earn CITI certification and are trained as usability session facilitators
  • Robust usability sessions of our online library tools each month. Huzzah!
  • Establishment of an engaged Library Student Advisory Board for User eXperience (LabX) which meets monthly and provides invaluable feedback
  • Improved wayfinding signs in McCabe Library with the help of my colleague, Visual Initiatives + Exhibitions Librarian Susan Dreher

Some goals for this coming year:

  • Communicate my work more effectively through more frequent, shorter blog posts, posts to our library staff email digest, flyers, and an occasional newsletter (maybe…)
  • Collaborate with the AUX Committee to formalize our Assessment Plan based on our operational goals and objectives
  • Cultivate the use of our Data Dashboard for more cross-pollination of our data stories
  • Seek out relationships in the college community outside the libraries. I’m looking at you Office of Institutional Research, Communications Office and Center for Innovation and Leadership!
  • Continue to work on the ACRL Proficiencies for Assessment Librarians and Coordinators
  • Build relationships with librarians outside of Swarthmore College by becoming more active in professional organizations

Happy weekend – stay cool and hydrated!

MISO

The Libraries and ITS participated in the Measuring Information Service Outcomes, er, the MISO survey this year. MISO is a web-based quantitative survey designed to measure how faculty, students and staff view library and computing services in higher education.

The two departments first participated in this nationally run survey in 2016 and decided to repeat the survey every three years. We hope to learn from longitudinal data and want to be aware of any trends.

The Libraries were particularly interested in the 2019 survey results, since the new version of our catalog, Tripod, launched in December and the survey ran shortly afterward, in February. We knew there were hiccups in the Tripod launch and were eager to compare the community responses about the library catalog to their responses in 2016.

Interestingly, while we have much still to learn from MISO results, there were no statistically significant differences in how our community views our services since 2016, including the catalog, Tripod. Yet, we are aware of deeper questions to ask of the data and of areas for which additional research is needed.

For example, we learned that a higher percentage of faculty do not use Tripod or library databases than we would expect and further discovered that the number of faculty who do not use these resources has increased from 2016 – 2019. The number of faculty who say they “do not use” Tripod rose from 7.4% in 2016 to 12.9% in 2019. We’re in the process of cross tabulating the data with help from colleagues in the Office of Institutional Research in order to learn the extent to which academic division or other factors may play a role in the increase.

Faculty Survey: Skill Level with Tripod

Another area for further exploration is the number of faculty, students and staff who say they feel “not informed” or only “somewhat informed” about library services. To help us communicate what services we offer more effectively, we need to first learn what people think of when they see the term “library services.” Do they think about circulation, or perhaps access to online resources? Naturally, we’d prefer a larger percentage of our users feel “Informed” about our services, so we’re thinking of ways to address this.

Student Survey: How informed do you feel you are about library services?

On a wonderful positive note, our faculty, staff and students continue to rate their satisfaction with librarians very highly, between 3.8 or 9 on a 4 point scale, for being friendly, knowledgeable, reliable and responsive. Thank you Swarthmore community – we love you right back!

Staff Survey: How satisfied are you that the library circulation staff are: Friendly, Knowledgeable, Reliable and Responsive?

Serendipity…

…the occurrence and development of events by chance in a happy or beneficial way.

Several years ago, I was invited to work with a Swarthmore College alumnus to learn about website user experience and to use what I learned for our library catalog redesign project. The alumnus had offered his time to the College and I happened to be available and interested in a new project. While the redesign project was compelling, the biggest impact for me was the revelation of a new focus in librarianship.

My exposure to the power of observing how users navigate our tools created a deep interest, not only in how users interact with the shared, online tools we provide, like Tripod (our library catalog) and Research Guides, but also in the broader contest of the role of the libraries in the research process.

How do our users approach their research? How do they navigate finding what they seek? How can we best support this process? How do information architecture, web content, writing for the web and web design work together to propel research forward, or alternatively, raise barriers or introduce pain points that get in the way? How do we provide the best environment for discovery? What role do library spaces and services play in the research process?

One of my goals as the Assessment and User Experience Librarian @ Swarthmore is to communicate regularly about what I’ve learned from user experience and assessment projects. I’d like to share our practices for user research, be able to demonstrate why it’s important and what impact it has on our systems and spaces.

To close my first post, I’ll share the graphic, below, created by library UX practitioner Andy Priestner and first unveiled at his talk at the 2017 UX in Libraries conference in Glasgow. For additional fun, you may follow the prompts at this site created by another library UX practitioner, Vernon Fowler, and score where you think your library falls on these continua. For even more fun, try it as a group and talk about where you would place your library.