Streaming Film Assessment

This year, the libraries are doing an assessment of our film collections in order to better understand how changes in the film industry and technology, particularly streaming options, are impacting the libraries’ materials budget.

We’re investigating everything we can think of:

  • Purchase requests
  • Statistics from our licensed streaming packages
  • Statistics for video placed on reserve and checkouts
  • How many “ripping” requests we receive (to make DVDs available for streaming) and how much time this requires
  • Film related workflows from request to delivery
  • What supports and services for providing film might be duplicated or supported by our campus partners, academic technologists in ITS and the Language Resource Center.

So far, we’ve:

  • Gathered statistics on:
    • Our streaming packages
    • Films placed on Reserve
    • Netflix use
  • Spoken to colleagues in the Language Resource Center to learn about all the ways they support the use of film in the curriculum
  • Devoted a student Library Advisory Board (LabX) meeting to get feedback from students about their film use in order to learn:
    • How often they use the collections for study breaks and entertainment
      • whether they consider checking out DVDs
      • what the libraries might do to make watching DVDs more convenient
    • How they view films required for class:
      • what happens if their schedule prevents them from viewing a film outside of class at a time the professor arranges?
      • how they watch a required film that may be available only through a platform the libraries are unable to support, like Amazon Prime or Hulu?

Our next step will be to send a short survey to about twenty professors who are known to use a lot of film in their courses in order to learn:

  • About how many films they may require in a film-heavy course
  • Whether their need for film is growing
    • Are standard courses taught from year to year impacted?
    • Does film play a larger role in newer courses?
    • Does the proliferation of available content suggest growing need?
  • What drives their preference for format?
    • Availability?
    • Accommodation (captioning, multi-language subtitles)?
    • Copyright?
    • Big screen vs laptop?
    • Picture and sound quality?
    • Convenience of delivery?
  • Are provisions made for students to view film outside of class time?
  • What if a film they want to include is available only on a platform the library cannot support (such as Amazon Prime or Hulu?) What accommodations do they make for students?
  • How might the libraries best support their need for film?
    • How might we improve the request process?
    • Should we investigate the feasibility of providing students with Amazon Prime, Hulu or other streaming platform accounts?
    • What other ideas and opportunities could there be?

We hope our assessment will inform:

  • Whether the evidence we gather supports advocating for more dollars in the budget, or guides difficult choices in order to balance the budget and curricular need
  • How we might improve our workflows from purchase request to film delivery
  • Hidden opportunities that may improve our collections and services in the future

If you have ideas of other avenues to explore, please let anyone on the AUX Committee know! (Assessment and User Experience)

Alison Masterpasqua, Amy McColl, Peggy Seiden, Barb Weir, Mary Huissen, Pam Harris, Pat O’Donnell – with much support for this project from Jessica Brangiel and Nabil Kashyap

Wrap up the semester and tie it in a bow

I recently hosted a usability drop-in session supported by one of my student usability workers. We ran two navigational tasks on our catalog site: 1) finding reserves and 2) finding links to archival and digitized collections.

We used A/B testing for the second task; I ran the sessions using a test instance of our site on which I had placed links to other collections on the catalog home page, while the student assistant ran the tasks using the live site, where the links are behind a “more” button and a curtain.


We ran a reserves task last spring within a few months of launching Alma/Primo and learned that users were not able to complete the task successfully. At all. While I do not like to admit that we’ve done nothing in the meanwhile to make it easier for users, this time the results were quite different:

  • 66% of users in our session successfully found reserves by searching by either the course name or course code and limiting their search with the Course Reserves scope.
  • 22% searched by course title and used facets to find reserves.
  • 11% of users looked for a top link to Reserves and they were not successful.

Huzzah! The only explanation I have for greater success is that once we knew addressing the problem with a customized solution would take months, we communicated the out-of-the-box path to Course Reserves through a variety of channels. Perhaps the communication had an effect, or perhaps users have become more familiar with the system and have learned the pathway as it is without customization.

Despite greater levels of success, since the users who struggled with the task continue to look for a top link for Reserves, the TriCollege Discovery and User Experience group (DUX) will discuss customization options.

Coincidentally, I’ve been conducting an environmental scan of other Alma/Primo sites within the Oberlin Group of colleges and found that Albion, Hope and Kalamazoo Colleges have put a solution in place that we’ll evaluate. We may also explore the solution that Whitman College Library created by customizing two separate scope options within the main search bar.

Links to Other Collections

I chose this task because I’m motivated to reduce the number of top links in Tripod. Our users naturally look to the top links, but when they click on the more button (identified by three horizontal dots)

the more button

they are faced with an off-putting curtain of thirteen links.

curtain of 13 links

The reason 13 appear is that the system repeats the first 6 links which already appear at the top of the home page and adds the rest. Since I haven’t found anyone who has managed to eliminate the redundancy, I’d like to reduce the overall number of links to no more than six in order to make the more button unnecessary.

The results of this usability session pointed to a solution (already implemented) that replaces the final four links behind the curtain with highlighted space on the catalog homepage.

Each user whose session took place on the test instance with this card in place found the collections easily, while users exploring the live site had trouble.

elevated to the home page

I’m pleased with the progress we are making with usability at Swarthmore. I’m grateful for a great team of student assistants who support the mechanics of usability sessions and for receptive colleagues.

Yet my experience tells me that usability results often point to larger conceptual questions in addition to the web fixes that might be addressed by our developers and this can be disheartening.

It’s great if developers implement changes that solve usability issues, but library research and catalog user experience can go beyond things that might be addressed with code.

  • How can we build contextual information about how to do research, where to find appropriate resources and how libraries and databases function into the catalog?
  • How can our web pages, the catalog and LibGuides play together more intuitively to the benefit of our users?

Library Website Usability Sessions

Student library UX workers and I facilitated 14 usability sessions on the Libraries’ website in late September in order to learn whether changes made to the site over the summer are intuitive for users.

We began each session by asking basic demographic information, the user’s year in school or college affiliation and general areas of academic interest. We also asked if they had ever used the site before and followed up by asking “why-or-why-not?” (And it was at this point we learned that many Swarthmore students think that the Tripod catalog and the Libraries’ website are the same thing…)

We ended each session by asking about impressions of the site, what works, what does not and what could make it better.

In between these opening and closing questions, facilitators used scenarios which prompted users to accomplish at least one (but not all) of the following tasks using the site:

  • Find different kinds of research help
    • Find citation help
  • Get access to popular resources: the New York Times and The Economist
  • Find space for quiet study and group study spaces
    • Reserve a group study
  • Borrow Technology
  • Discover what online resources are available to Alumni
  • Find information for visitors: hours, use of computers, printers, scanners, and group study rooms.

Here is what we learned:

Research Help + Citation Help
We captured 9 sessions using the Research and Citation Help scenarios. While eight of the nine users ultimately had success finding at least one kind of research help using the site, only one user, a librarian, began at the Get Help page.

Alternate successful paths to research help were via:

  • a link to Research Guides within the Tripod search widget
  • the Chat button
  • Support for Research + Teaching>for Students

Duplicate paths to information can be great, but since students did not use (or maybe see or understand) the Get Help link, we’re considering adjustments. We may change the label (after testing out suggestions for something more intuitive) and we may consider integrating the Get Help content with overlapping content on the Support For Students page.

An unexpected outcome of this task was to learn that many students equate finding resources with getting research help. If a user begins with this understanding, choosing the “Find Resources” link instead of “Get Help” makes perfect sense, but the result is that students overlook or remain unaware of the variety of library support available outside of the Tripod catalog.

More than half of the users began this task with either a Tripod catalog search or via the Find Resources link in the site navigation instead of Get Help. A majority of them probably would not have discovered information about the kinds of research help from Librarians or Guides for which the scenarios were designed except for being encouraged to seek it due to the context of the usability session.

There was only a 50% success rate for the citation help task. If the user began seeking research help by looking for resources, they were not successful in finding citation help on the site.

Additionally, these unsuccessful sessions demonstrated that anyone who searched the entire College website (via the site search magnifying glass) using the term “citation” or “citations” was not successful. Instead, top results for these queries on the College site are for the Writing Center, academic departments, and even the Public Safety Department. (Parking tickets!)

We’re adjusting our headings in hopes of having citation help from the libraries appear as a top result in a search of the College website, but the problem raises an additional structural issue that affects library site usability: a lot of library help is in LibGuides, and as a result is not exposed to a site search within

The citation information on the Writing Center website is quite good although not as thorough as what the Libraries’ offer (via LibGuides.) Is there any opportunity here to collaborate with the Writing Center by linking to one another’s information?

We’re also exploring the feasibility of a technical solution that may pull LibGuides content into a site search.

Find Popular Resources
We captured 7 sessions of this task beginning with getting access to the New York Times. Three users had success via the Popular Online Resources page. Another three tried a Tripod search for the Times, but only one of these had success getting access. One user could not get access to the Times at all.

The Popular Resources page is a help to those who find it, but it’s buried in the navigation. A hover menu would be a big help.

Library Space
We ran this task in four sessions and each user had success.

Borrow Technology
We ran eight instances of this task with mixed results. The information is on the Support for Students page, but underneath the rather confusing heading “Need Something?”

Since many users were looking for the information on the Borrow + Request page, I added it there and it had an immediate effect.

Overall, six users were able to find the information, (some after I had changed the site) but three were unable to find it even after the change.

A common response to this scenario was “I already know the libraries lend chargers and I would just go to the desk and ask for one.”

Alumni + Visitor tasks
Users were mostly successful with these tasks, but whenever they were not, it was due to the information being organized into Drupal link-list-panes. If the user chose the wrong link, they did not return to the list in order to accomplish the task.

As a result, we have plans to reorganize the information out of the current combined page into separate pages, one for Alumni and another for Visitors.

Larger Issues
Many of our users are confused by the interplay of the Libraries’ website and the Tripod catalog. Additionally, usability of our website suffers from being unable to provide hover menus that expose the extent of our content at a glance.

While we may be able to address Tripod issues and branding with our Trico partners, the path to resolving functional and structural limitations at the College is not as clear. Meanwhile, we’ll continue to make changes to the navigation within the structure we have and continue testing for usability.

Space exploration, or, UX on-the-fly

This semester I’ve noticed frequent evidence of science-y stuff happening on the big whiteboard that’s outside my office as I arrive at work each day. The content is striking because my office is in the Humanities and Social Sciences Library and we have a recently renovated Science Library nearby.

I’ve noticed groups of students working on science projects in this space throughout the day, so decided to start asking them why they are choosing to work here instead of the science library.

The first student I asked was working out a chemistry problem on the whiteboard. He said he chooses the space because it’s a little quieter and because the science library “has a pretty overwhelming STEM vibe” that makes him uncomfortable.

But my direct question seemed to make him feel uncomfortable too and I worried later (especially after noticing the beautiful chemistry symbols erased from the whiteboard) that I may have made him feel as if he didn’t belong. Note to self: leave brief reply cards in this area and collect the information anonymously and more voluntarily! I’d love to know more from this student. What did he mean by the uncomfortable STEM vibe? After all, chemistry is a STEM subject.

I next approached a group of three students hard at work on a science project in this same space and inquired what draws them here instead of the Science Library. Their answer was different: they meet together every Wednesday morning and come from different places. This space is the most centrally located.

However, they added that the group spaces in the Science Library are not as conducive to their work: spaces there are more open and the group feels more exposed than they like, plus it’s generally louder over there.

This particular spot is about to disappear as a result renovation this spring (hence the empty stacks) but it’s interesting to note the aspects that make it appealing:

  • it’s centrally located
  • quiet-but-not-silent
  • has a large whiteboard
  • is enveloped by tall stack ranges on either side of a spacious work table which help create privacy and a more enclosed feeling (maybe like a little hug from the libraries? I can hope!)

As we think about library space, we gather lots of quantitative data: how many students use a space, what they use the space for, what kinds of equipment they use, what times of day and night are more crowded, how much space we need for collections, how to balance desires for quiet with the need for group work space. It’s important to remember to gather qualitative data from a broad spectrum of users too.

Now about those anonymous, voluntary reply cards…

Small victories

A few weeks ago I overheard someone say “The month of August is like one, long Sunday evening” and I smiled, knowing exactly what she meant. It’s a bittersweet time saying goodbye to a more relaxed summer pace in the libraries mixed in with our preparation for saying hello to the official start of the semester, which is still a week away. But if the analogy holds true, I believe the weekend is over and it’s already Monday morning despite the calendar indicating it’s still August!

Student athletes who compete in autumn sports arrived last week and international students arrived several days ago along with cohorts of students whose jobs facilitate the beginning of the semester or require intensive training (resident advisers, student academic mentors, orientation leaders and the like.)

As I arrived at work this morning, it was abundantly clear that today is “move in day” for first year students. Ready or not, we’re off to the races!

My work is not greatly affected by the seasonal shifts of the academic year. It’s true there are fewer library users in the summer months, but I remain as engaged as ever as an advocate for their ideas and improved experiences.

But even though I’ve been at this for awhile, it continues to surprise me how much time can elapse from first learning the germ of a good idea for an improved service to turning the idea into reality.

Almost one year ago, students in the Library Advisory Board and User Experience Group (LabX) suggested we post guides to LC class subject areas within the stacks at the point of need, for example placing a small poster at the beginning of the “As” saying: A class is for General Works, Encyclopedias, Dictionaries, etc.

This was a simple idea I thought would be easy to turn into reality. But the pathway took more twists than I imagined and it started with library staff discussions about instead providing a guide to the entire range of LC subjects centrally on each floor and making attractive LC Class subject guide bookmarks freely available. That is, er, not providing the stack guides at the point of need or curiosity.

This example of advocating for users is certainly a low stakes one; moving forward with the idea was not expensive or disruptive and the impact may not be measurable beyond any appreciation expressed by the student group which offered the suggestion.

But it illustrates that user advocacy can be tricky, particularly in an institutional environment that has a heart for consensus and a decision making structure that is sometimes opaque.

I have been grateful for this low-stakes project because it has provided me with needed practice to grow more confident professionally despite push back from trusted and well meaning colleagues.

I hung the stack guides up at the end of yesterday and felt victorious.

Best wishes for the new academic year everyone!

Dog Days of Summer

It’s a Friday afternoon in the midst of a punishing heatwave in Swarthmore during the dog days of summer…

Photo by Paulette Wooten on Unsplash

…we’re just past the midpoint of the calendar year and a new academic year is on the horizon. What a good time to take stock of some of the goals we have achieved so far and to think about new ones for the upcoming year.

Since starting in this position, I’m proudest of these things:

  • IRB approval for usability testing
  • Approval to hire students at the highest pay grade who earn CITI certification and are trained as usability session facilitators
  • Robust usability sessions of our online library tools each month. Huzzah!
  • Establishment of an engaged Library Student Advisory Board for User eXperience (LabX) which meets monthly and provides invaluable feedback
  • Improved wayfinding signs in McCabe Library with the help of my colleague, Visual Initiatives + Exhibitions Librarian Susan Dreher

Some goals for this coming year:

  • Communicate my work more effectively through more frequent, shorter blog posts, posts to our library staff email digest, flyers, and an occasional newsletter (maybe…)
  • Collaborate with the AUX Committee to formalize our Assessment Plan based on our operational goals and objectives
  • Cultivate the use of our Data Dashboard for more cross-pollination of our data stories
  • Seek out relationships in the college community outside the libraries. I’m looking at you Office of Institutional Research, Communications Office and Center for Innovation and Leadership!
  • Continue to work on the ACRL Proficiencies for Assessment Librarians and Coordinators
  • Build relationships with librarians outside of Swarthmore College by becoming more active in professional organizations

Happy weekend – stay cool and hydrated!

Bench-marking the user experience from one interface to another

As we settle in becoming more accustomed to the newest version of Tripod, it is useful to look back and remember the massive amounts of work that went into system migration.

A year ago, most of my colleagues were heavily invested in necessary behind-the-scenes planning and technical work, while I focused on how best to address the approaching changes to the public-facing user interface with our users.

Many of us have opinions about how we’d like websites to look and feel in addition to how we want them to work, but the best way to know whether an interface functions well aside from visual design is through usability testing. Are users able to accomplish the tasks for which they came to the site?

In preparation for system migration, I held usability sessions with the now-retired version of Tripod in order to be able to compare results for how users accomplished essential tasks there with what we would learn from usability sessions in the new system.

What tasks are essential to users of a library search interface? We decided to test the following:

  • Account login: including saving items of interest to lists, sharing them with others or sending them to yourself via email or text
  • Known-item searches: placing a hold or a request from another Trico Library, and if the item is not available, taking the next step toward E-ZBorrow and ILL
  • Broad searches followed by use of facets (the “Refine your Search” options at the left of the screen) to limit results
  • Finding items on Reserve for class

In the fall of 2018, users of “old” Tripod had success logging into their accounts, adding items to lists of favorites, and emailing or texting the information to themselves or a research partner.

They were able to find specific titles and to place holds and Trico requests. And despite some participants having no experience with E-ZBorrow or ILL, they had heard about these services and were able to navigate to the appropriate links.

But despite the near ubiquity of facets on monster, commonly used websites like Amazon, many Swarthmore students either have not noticed them in Tripod or have not used them. This task caused trouble.

And every user struggled to find items on Reserve for a specific course using Tripod.

These results were not surprising. Library searching can be complex, we know there are students who do not use Tripod and novice researchers may struggle with even the most intuitive interface. The purpose of these sessions was to set a baseline for comparing results between old and new Tripod.

So about those! Results from post-launch usability sessions on “new” Tripod early in 2019 were largely the same: students navigated intuitively to the basics of what they could accomplish while logged into their account, they found relevant results for their known-item searches and pathways to place requests and to borrow beyond Trico. They also overlooked facets and really struggled to find Reserves!

We’ve already made improvements to the Tripod interface and have prioritized making the pathway to Reserves more intuitive before classes begin in the Fall.

Regular usability sessions are vital to improving user interfaces and the libraries run them monthly during the school year. Keep your eyes out for calendar announcements, stop by and try it! The “tests” are of our online interfaces, not users! It’s fun, you’ll be rewarded with a home-baked treat, and best of all, you’ll help to keep making Tripod (or the website, or Research Guides) better!


The Libraries and ITS participated in the Measuring Information Service Outcomes, er, the MISO survey this year. MISO is a web-based quantitative survey designed to measure how faculty, students and staff view library and computing services in higher education.

The two departments first participated in this nationally run survey in 2016 and decided to repeat the survey every three years. We hope to learn from longitudinal data and want to be aware of any trends.

The Libraries were particularly interested in the 2019 survey results, since the new version of our catalog, Tripod, launched in December and the survey ran shortly afterward, in February. We knew there were hiccups in the Tripod launch and were eager to compare the community responses about the library catalog to their responses in 2016.

Interestingly, while we have much still to learn from MISO results, there were no statistically significant differences in how our community views our services since 2016, including the catalog, Tripod. Yet, we are aware of deeper questions to ask of the data and of areas for which additional research is needed.

For example, we learned that a higher percentage of faculty do not use Tripod or library databases than we would expect and further discovered that the number of faculty who do not use these resources has increased from 2016 – 2019. The number of faculty who say they “do not use” Tripod rose from 7.4% in 2016 to 12.9% in 2019. We’re in the process of cross tabulating the data with help from colleagues in the Office of Institutional Research in order to learn the extent to which academic division or other factors may play a role in the increase.

Faculty Survey: Skill Level with Tripod

Another area for further exploration is the number of faculty, students and staff who say they feel “not informed” or only “somewhat informed” about library services. To help us communicate what services we offer more effectively, we need to first learn what people think of when they see the term “library services.” Do they think about circulation, or perhaps access to online resources? Naturally, we’d prefer a larger percentage of our users feel “Informed” about our services, so we’re thinking of ways to address this.

Student Survey: How informed do you feel you are about library services?

On a wonderful positive note, our faculty, staff and students continue to rate their satisfaction with librarians very highly, between 3.8 or 9 on a 4 point scale, for being friendly, knowledgeable, reliable and responsive. Thank you Swarthmore community – we love you right back!

Staff Survey: How satisfied are you that the library circulation staff are: Friendly, Knowledgeable, Reliable and Responsive?


…the occurrence and development of events by chance in a happy or beneficial way.

Several years ago, I was invited to work with a Swarthmore College alumnus to learn about website user experience and to use what I learned for our library catalog redesign project. The alumnus had offered his time to the College and I happened to be available and interested in a new project. While the redesign project was compelling, the biggest impact for me was the revelation of a new focus in librarianship.

My exposure to the power of observing how users navigate our tools created a deep interest, not only in how users interact with the shared, online tools we provide, like Tripod (our library catalog) and Research Guides, but also in the broader contest of the role of the libraries in the research process.

How do our users approach their research? How do they navigate finding what they seek? How can we best support this process? How do information architecture, web content, writing for the web and web design work together to propel research forward, or alternatively, raise barriers or introduce pain points that get in the way? How do we provide the best environment for discovery? What role do library spaces and services play in the research process?

One of my goals as the Assessment and User Experience Librarian @ Swarthmore is to communicate regularly about what I’ve learned from user experience and assessment projects. I’d like to share our practices for user research, be able to demonstrate why it’s important and what impact it has on our systems and spaces.

To close my first post, I’ll share the graphic, below, created by library UX practitioner Andy Priestner and first unveiled at his talk at the 2017 UX in Libraries conference in Glasgow. For additional fun, you may follow the prompts at this site created by another library UX practitioner, Vernon Fowler, and score where you think your library falls on these continua. For even more fun, try it as a group and talk about where you would place your library.