The user interview is our chance to learn about our users directly by asking them questions and observing their behavior and environment. This Interview Protocol outlines the procedure and methods for conducting user interviews.

Items Needed for an Interview

Interviewer Training - review before conducting interviews for information on interviewee recruitment, best practices for interviewers and note-takers and procedures for note-taking and debriefing.
Consent Form - each interviewee must understand and sign this before an interview.
Interview Script - introduction, questions and topics to cover in an interview.
(optional) Sample Interview Notes - have a look if you are wondering how to format your final notes.

What to do after completing an interview

Have a debriefing session with your interview team soon after your interview to review the interview, add to and clean up notes.
Upload your cleaned up, anonymized notes at the Interviewer Group Google Group.
Email the Data Analysis Group with 1-hour times over the course of a week that your interview team will be available to have a phone conference to review the interview. Depending on the notes and the complexity of the interview, we may not need to meet for the full one hour.

Interviewee Consent Form

Download Consent Form

Confidentiality and consent forms are standard practice for research studies that involve direct interaction with people. They serve to make sure that interviewees understand what they are getting into, can choose to discontinue their participation at any time they see fit and that the data they provide will be kept in the strictest confidentiality. The form also helps set the interviewee at ease to provide honest, critical answers to questions that they may normally gloss over if they knew their answers were going to be made more publicly available.

All our interview teams need to carefully abide by the directives set forth in our confidentiality and consent forms. In summary, these directives are:

Raw data from the interview will not be shared with anyone outside the interview team.
The interview team may share the data from the interview with other research colleagues, but all personally identifiable information about the interviewee or anyone else the interviewee personally identifies will be removed. There should be no way the data can be linked back to the interviewee.
The interviewee is free to deny responding to any questions or prompts presented by the interview team.
The interviewee is free to end the interview at any time.

IRB Approval

An Institutional Review Board (IRB) is a committee that has been formally designated to approve, monitor, and review biomedical and behavioral research involving humans with the aim to protect the rights and welfare of the research subjects. Your institution likely has an IRB and needs to be consulted when conducting different types of research on human subjects.

Because our work for this project is related to the design of a new system, we are not asking interviewees to expose themselves to any risk or harm outside of that encountered in daily life and we are not intending to formally publish (i.e. in a journal or archived conference proceedings) our findings, we can conduct our work without IRB approval.

You may be interested, however, in seeking IRB approval at your institution for your user interviews. This would allow you to do your own analysis of the data you have collected and publish the results. This project has qualified as exempt at a number of institutions. The IRB application for exempt projects is usually faster than standard IRB applications. For more questions on IRB or to see sample exempt applications, please email us.

Interview Script

Download Interview Script

The Interview Script is a document that the interview team can follow to conduct the interview. It contains an introduction, the questions to be asked of the interviewee and topics to focus on in observation.

Interview Introduction

A good introduction before beginning the interview is necessary to make sure the interviewee understands the goals of the interview, the relationship between the interviewee and the interview team and how best to respond to questions and other prompts.

The Interview Introduction is included below and is part of the Interview Script.

The main focus of our interview today is to understand more about why and how you use scholarly resources in preparing and conducting your courses.

We consider you the expert at your work so there are no wrong answers to any of our questions. While you answer questions or guide us through tasks, please focus on the details of how you actually do your work. It may help to think about the last time you performed the task and explain it to us as if we are going to need to perform the task just as you did. Please feel free to be honest and critical even if the way your work actually gets done is not the way it should be done - we are testing the systems you use, not you, and everything you tell us is strictly confidential.

Any questions before we begin?

Interview Questions

Interview Phases

The interview questions can be broken down into the following sections:

Warm-up: getting to know the instructor a little bit better and giving them some simple questions to get them feeling confident.
General Questions: learning a little bit more about the goals, attitudes and activities of the instructor with more open-ended questions.
Observation: focusing in on a specific activity and having the instructor think aloud, sharing details of why and how they perform tasks that comprise an activity.
Follow-up: asking questions to clarify missing, ambiguous or confusing information from observation.
Wrap-up: asking questions about the interviewing process, anything they would like to add and if it would be all right to contact them with follow-up questions or the opportunity to give feedback on prototype designs.

Format of a question

The format of the questions in our Interview Script are as follows:

[type] Question to be posed to instructor. Topics to cover, potential follow-up questions, focusing questions.

Example:
[Motivation] What do you enjoy most about your job as an instructor? Why? What activities do you always tackle first?

Types of questions:

[Variables] - Verifying demographic and behavioral variables
[Mental Model] - How does the instructor think about work, resources, services & systems?
[Activities] - What kinds of activities does the instructor perform? Do they occur regularly or occasionally?
[Motivation] - What are the instructor's goals? What does the instructor enjoy doing?
[Opportunities] - What are the instructor's pain points? What do they avoid?
[Interactions] - Who or what does the instructor consult or work with?
[Process] - How does the instructor make decisions and perform tasks?

Warm-up Questions

[Variables] How long have you been at <instructor's institution>?

[Variables] About how many faculty are in your department?

[Variables] How much time do you commit to teaching versus other job duties?

[Variables] Do you primarily teach small (up to around 30 students) or large (much more than 30 students) courses?

[Variables] Do you primarily teach undergraduate or graduate courses?

[Mental Model] How do you define scholarly resources? What kinds of things do you consider scholarly resources?

[Mental Model] Are there other resources you use regularly for your courses?

General Questions

[Activities] Could you please describe a typical day for you at work? Which activities occur regularly? Which activities occur only on occasion? How does course preparation and conducting a course fit in?

[Motivation] What do you enjoy most about your job as an instructor? Why? What activities do you always tackle first?

[Opportunities] What do you enjoy least about your job as an instructor? Why? What activities currently waste your time?

[Activities] Can you tell us more about the course you are currently most involved with? (If there is no definitive course: please tell us about a course you are currently teaching or, if you are not currently teaching, a course you most recently taught.)

[Activities] Is this a course you regularly teach?

[Variables] How many students are there?

[Variables] Are there multiple sections?

[Interactions, Process] Do you work with other instructors (including students) to prepare for the course before the start of the course? What do you do together?

[Interactions, Process] Do you work with other instructors (including students) to conduct the course once it has begun? What do you do together?

[Activities] In which course activities do you make use of scholarly resources?

[Process] For each activity, what scholarly resources are used?

[Process, Motivation] For each activity, how do you choose scholarly resources? How do the scholarly resources add value to the activity they are a part of, if at all?

Observation

In this part of the interview, the interview team acts as apprentices observing as the instructor walks through an activity, interrupting with questions only occasionally. The goal is for the interview team to be able to re-create the work of the instructor. The activity of interest should be one that the instructor has said they perform regularly in preparing or conducting their course and scholarly resources are used in the activity.

For example, the instructor may regularly use journal articles as reading assignments when building a syllabus in preparation for a course. Given this example, the interview team may request the instructor give a running description of how they set up a syllabus:

We would like to observe as you give us a running description of how you set up the syllabus for your course. Please walk us through this activity just as you performed it last, thinking aloud and letting us know why you are doing what you are doing. Think of yourself as a master and us as your apprentices: we want to understand how you do your work so that we may re-create it.

Be sure to have the instructor focus on just one activity, think aloud and actually complete the activity. If time does not allow for the completion of the activity, be sure to ask how the instructor would finish the activity.

Areas to focus on during observation:

Sequences - in what order is the instructor performing tasks? Are there interruptions? Are they doing more than one thing at a time? We want to be able to create rich workflow models (exactly how the user completes certain tasks) from this kind of data
Tools - what is the instructor using to perform their work?
Problems - what is frustrating or unexpected for the instructor?
Mental models - how does the instructor think about their work? What expectations do they have?
Interactions - do they consult different people, reference materials, processes? If so, what for?

Interrupting the instructor during observation:

The interview team should interrupt the instructor only in the following circumstances:

The instructor is doing a lot of work without speaking about it – Ask the instructor questions about what they are doing and remind them to give a running description of what they are doing so that you, the interview team, may be able to re-create their work.
The instructor is giving very general or broad details about an important task or action – Ask the instructor questions about what they are doing and remind them that you are looking for details of their work so that you can re-create it. Remind them that they can recount the details of the last time they went through this activity to make it more concrete.
It is unclear what the instructor is talking about – the instructor may use jargon that the interview team is unfamiliar with. As an apprentice, it is important to know what you are dealing with and it is all right to interrupt the instructor to ask a clarifying question when you do not know what they are talking about.

Follow-up Questions

In this part of the interview, the interview team takes time to reflect on the observation:

Are there gaps in the process the instructor has described to you? Now is the time to investigate and make sure all the steps involved in completing the activity are recorded. The interview team may end up delving back into observation mode as the instructor remembers something they forgot to mention and begins another description of tasks and actions.
Are there other parts of the process that are unclear? Now is the time to clarify.
Were there instances where the interviewer or note-taker had a question, but did not ask because it would have adversely interrupted the instructor? Now is the time to ask these questions.

Wrap-up Questions

Give the interviewee a big Thank You!

What did you think of this interview? Anything you did not understand? Anything that made you anxious?

Is there anything else regarding use of scholarly resources in preparing or conducting your courses that you would like to add?

Would you be willing to be contacted with follow-up questions?

Would you be willing to be contacted about providing feedback on prototypes we design?

Note-taking

View Sample Interview Notes

Guidelines:

Try and capture every word the user says!
Take notes directly on a laptop if the note-taker is a fast typist
- Taking notes by hand (can be useful in creating diagrams) or making an audio recording are also suitable options if the note-taker is more comfortable with these methods. We do require that interview notes shared with the rest of the group be typed up. Taking notes by hand or audio recording and then transcribing them may take a significant amount of time.
Abbreviations are helpful, paraphrasing is OK
- Try and record especially poignant quotes directly and mark them as quotes (i.e. "The library is too overwhelming to approach cold.")
Raw notes do not need to be perfect
- Clean up notes during debrief soon after the interview
It's OK for the note-taker to ask to repeat something or ask for clarification (without interrupting the instructor too much)
- in between questions are a good time
If the interviewer comes up with new questions, record them in the notes

Post-interview Debrief

Arrange a debriefing session as soon as possible after the interview
Discuss and record your initial general impressions from the interview in the Debriefing Notes section at the end of the Interview Script
Save a second file (Save as...) as the "clean" version of the notes
Go through the notes removing any personally identifiable information (names, non-work related places, etc.)
- Names/places should be substituted with anonymous titles:
  - Gaurav Bhatnagar helped me ... --> [a faculty colleague] helped me ...
Go through the notes adding text to make the notes understandable by those outside of the interview
- Complete fragmented sentences or thoughts
- Expand abbreviations

After debriefing with your interview team:

Upload your cleaned up, anonymized notes at the Interviewer Group Google Group.
Email the Data Analysis Group with 1-hour times over the course of a week that your interview team will be available to have a phone conference to review the interview. Depending on the notes and the complexity of the interview, we may not need to meet for the full one hour.