User interviews were held with four faculty at the University of Michigan early on in the Library & Sakai 3 design effort (June 2009) to gain some general user data and demonstrate the proposed design process. The focus of the interviews was to learn more about the goals, frustrations and workflow of faculty around finding, using and sharing scholarly resources for teaching, learning and research.

Selecting Interviewees

To select a range of interviewees that would be as representative as possible, we solicited faculty that were at a training session on both library and CTools (UM's local Sakai instance) services. This ensured that the faculty we spoke with had at least some familiarity with digital library services as well as Sakai. The following behavioral and demographic variables were also considered in choosing interviewees and to gather information on during interviews:

• Time spent online
• Time spent as faculty/level of faculty
• Comfort level with placing course materials online
• Comfort level with technology in general (i.e. willingness to try/learn new systems)
• Teaching primarily large or small courses
• Teaching primarily undergraduate or graduate courses
• Preferred research methods and materials
• Commitment to teaching motivated by passion or by duty

The Interviewees

Four faculty members were interviewed:

Each interview was 30 to 60 minutes long. There was one interaction designer running the interview and taking notes on a laptop. Ideally, two interaction designers should conduct interviews - one to run the interview and one to take detailed notes.

After each interview, the interaction designer went through the notes cleaning them up by making them more readable (to be able to share with others) and adding in details that could not be captured earlier. This process also gave the designer a chance to reflect on the interview and identify emerging themes and new behavioral variables.

To help the designer remember the overall impression of the individual interview, short summary statements were developed describing those characteristics of the user that were particularly salient.

• F01: I'm an intermediate user. I'm comfortable with new learning and research technology, but do not always understand its value for me or enhancing my in-class experience.
• F02: I'm a late adopter. It takes me a long time to learn new technologies and then they change. Once a user, however, I'm enthusiastic and encourage others to convert.
• F03: I feel stuck. Students are demanding more online materials, but the more I put up, the fewer students I see in class. Those who do come are on their laptops emailing or on Facebook.
• F04: I'm a tech enthusiast. CTools is good at lessening my administrative burden, but certain tools do not give me the functionality or usage data I am looking for, so I've commissioned a few of my own online learning tools.

Findings

Behavioral Mappings

After interviewing, some of the initial behavioral variables we came up with before interviewing did not seem to apply while new ones were discovered. For example, 'Time spent online,' did not seem to apply because all faculty were spending a lot of time online. New behavioral variables were added in relation to technical aptitude and interest in recommendations from other colleagues and usage data from students.

The updated behavioral variables are as follows:

• Time spent as faculty
• Comfort level with placing course materials online
• Comfort level with technology in general (i.e. willingness to try/learn new systems)
• Skill level with technology in general
• Satisfaction level with CTools
• Level of reliance on help resources (colleagues, in-system support, staffed support, workshops, etc.)
• Level of interest in automatic recommendations from colleagues
• Level of interest in automatic student resource usage data
• Teaching primarily small or large courses
• Commitment to teaching motivated by duty/requirement or passion
• Common research methods and materials

All but one behavioral variable could be represented using a continuum that we could place each user along. The image below is the outcome of this activity (click to open full-size image in a new window).

Behavior Patterns and Supporting Evidence

Two main general behavior patterns emerge from the behavioral variable mapping and other data from the interviews:

• Pattern 1: The younger, more tech-savvy, more comfortable with online teaching material
• Pattern 2: The older, less tech-savvy, somewhat uncomfortable with online teaching material

Findings

Following is a detailed summary of what was found when analyzing the interview data and behavioral variable mapping.

Data that provides a basis for the 'Comfort level with technology in general' variable comes primarily from interviewees' description of the different systems they use, how often they use them, how quickly they are able to learn new technologies and their overall satisfaction or frustration with these systems. It takes the older behavior pattern much longer to trust and learn new technologies, and, by the time they learn them and feel comfortable, the introduction of an upgraded system with new rules to learn can be frustrating.

This distrust or misunderstanding of new technologies may relate back to the overarching needs expressed by all interviewees: to limit their time on administrative tasks and focus on research and teaching. In general, all interviewees are relatively satisfied with CTools primarily because it reduces their administrative burden in communicating with their students, managing student records and sharing resources with students. The less interviewees can quickly and clearly see the value of a new product in streamlining their work (either presented in marketing material or, more importantly, when they use the product for the first time) the more hesitant they are to invest time in learning how to use the product. This point is true across all interviewees: though F02 and F03 are older and require significant time in learning new technologies, F03 comments on not using many CTools tools because she just does not know how they will improve her teaching or in-class experience and F04 has gone so far as to develop his own online learning tools completely separate from CTools because CTools tools did not meet his needs.

Another important finding in regards to comfort in placing materials online and satisfaction with CTools relates to an overarching goal of all interviewees and potentially all faculty. As teachers, interviewees want students to show up to their classes, be engaged and leave with a deeper understanding and appreciation for their subject matter as well as tools and skills to independently study and further the state of the art. Interviewees are at a university that places high value on teaching and work within a system where feedback on their teaching impacts their career through raises, standing along the tenure-track and their perceived effectiveness as faculty. By placing more course materials online, faculty run the risk of seeing fewer students in class, as F03 has experienced first hand and, as a result, refuses to post her lecture slides online. The proliferation of laptops in class (which coincide with student demand for more online material) provide opposing opportunities for students to engage more actively in class (through detailed, organized and searchable notes, for example), or disengage more actively in class (through Facebook, chatting, or games, for example). F02 finds distraction more common in the larger, undergraduate required courses where students are there because they have to be rather than on their own accord. As an energetic and enthusiastic lecturer, he claims these courses are "boring" to teach. Both F01 and F04 have an interest in making their lectures more engaging by including more images and videos. Unfortunately, it can take a very long time to find good ones and they may only consume three minutes of class.

When running into problems with CTools or library systems, interviewees have a differing level of reliance on help resources. All interviewees attended a workshop on the new library reserves system and how to manage their new electronic reserves through CTools, so they are all somewhat comfortable with asking for help. Within the system, however, attitudes differ. F01 claims that because there are workshops, she should not ask questions when she has them. Because it is difficult to make it to all the workshops, she ends up not asking for help often and routinely does not experiment with features she does not understand, instead doing things manually. F02 claims he, "just fiddles around some more on [his] own," because technical help documentation is so tough to read. F04 is similar to both F01 and F02 in that he experiments on his own, but drops his exploration after a certain amount of time and does not ask for help often within the system. F03 relies on help a lot in the form of her own detailed notes on how she learned how to perform certain tasks.

All interviewees expressed the need for some automated feedback on what online learning materials students were actually using. F03 has made announcements about un-required articles that may be of interest of students. F02 has used the News tool to embed news sources from across the web in his CTools sites. These faculty, who are not generally tech-savvy, went out on a limb to try something new with CTools and had very little idea whether their effort was worthwhile. They mentioned asking about these special resources in class, but the response was not reliable. Both F01 and F04 expressed the same need in more general terms. If they were able to quickly see what students were using and what they were not, they could cut down time in deciding where to focus course improvements while increasing the relevance of their courses for students.

Similar to having usage statistics on student behavior, F01, F02 and F04 expressed interest in the collective wisdom of their colleagues at varying levels for different purposes to help them navigate digital learning tools and scholarly content. F01 is interested in making her lectures more engaging and knows there are a number of CTools tools available to her. She has no problem understanding them technically, but is not sure what she can do with them to make class more engaging and wonders how other faculty are using them in their courses. Both F02 and F04 are involved in cross-disciplinary work. They know the top databases and resources and how to navigate them for their own areas of expertise, but lack this expertise in other domains. When confronted with doing research in other domains or adding lecture content from other domains, both F02 and F04 look to colleagues in other departments for hints. For example, F04 teaches a musicology class on Jazz, but knows there are English Literature or African American Studies courses that are also discussing Jazz from a totally different, but very informative perspective that can add value to his Jazz class. F04 also mentions the difficulty new faculty have in navigating the vast amount of services and resources the UM Library has to offer and how veteran or retired faculty can cite their most used or recommended resources to alleviate this disconnect.

Interviewee motivations for teaching reveal interesting ways in which CTools and digital scholarly resources are perceived. F02 states that he enjoys lecturing because he is excited about the subject matter and he wants to share that interest with others. F03 mentions junior faculty that did not make tenure because they were very passionate about teaching and making "fancy PowerPoints". They were highly regarded faculty, but lost time on what F03 believes is their main focus on scholarship and publishing. Those interviewees that are more on the passion side of the teaching motivation spectrum tend to look at CTools and digital resources and wonder how they can best use them to better represent their subject matter and better engage students in appreciating subject matter. Those interviewees that are more on the duty side of the teaching motivation spectrum tend to look at CTools and digital resources and wonder how they can best use them to save time and carry on with their research. This spectrum can be somewhat misleading, however, because the two sides are not mutually exclusive. A faculty member can both have a passion for teaching as well as a sense of duty about being a strong teacher and researcher. A better focus for follow-up interviews may be to better understand if faculty spend more time on research versus teaching and why.

Finally, common research methods and materials for interviewees were discovered. All interviewees rely on the library's e-reserves service to have readings from books to journal articles available for students online. What was interesting to observe was the range of different ways interviewees found materials and how they thought about the library. Both F01 and F02 primarily use journal articles and described finding them "through the library." F01 actually starts at the library's catalog, Mirlyn (Ex Libris Aleph), and follows a link over to the metasearch engine, SearchTools (Ex Libris MetaLib), without noticing that she jumped between two completely separate systems. She even refers to what she uses as Mirlyn Power Search, even though 'Power Search' is a SearchTools feature, not a Mirlyn feature. F02 has bookmarked a number of the major psychology databases, such as PsychINFO. Although his bookmark to PsychINFO takes him directly to the PsychINFO native homepage through the UM Library proxy, he describes the journal as at the library. Because he found the link to the database at the library website and has some knowledge about the licenses the library has to allow him to search there, the database seems a part of the library. F03 and F04 have similar mental models about the library as a one-stop-shop for all things related to scholarly materials. Unfortunately, the library is not structured in this way and is rather a conglomeration of a number of different shops (i.e. catalog, articles and databases, e-journals, research guides, citations are all separate entities). There is a mental model diagram of this discrepancy in the Other Models phase Example. F04 finds Google (and YouTube) to be the best place to go to find the multimedia-rich content he is looking for. He knows, however, that once copyright starts impinging on his use of these "free" services, the library will be the place he turns to for finding this kind of material, hopefully just as quickly as he finds it through Google.