The user interview is our chance to learn about our users directly by asking them questions and observing their behavior and environment. This Interview Protocol outlines the procedure and methods for conducting user interviews.
...
| Section | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Interviewee Consent Form
| Tip |
|---|
Confidentiality and consent forms are standard practice for research studies that involve direct interaction with people. They serve to make sure that interviewees understand what they are getting into, can choose to discontinue their participation at any time they see fit and that the data they provide will be kept in the strictest confidentiality. The form also helps set the interviewee at ease to provide honest, critical answers to questions that they may normally gloss over if they knew their answers were going to be made more publicly available.
...
You may be interested, however, in seeking IRB approval at your institution for your user interviews. This would allow you to do your own analysis of the data you have collected and publish the results. This project likely qualifies has qualified as an exempt IRB case, so IRB processing may be a far faster process than normal. More information on IRB processing coming soon ..at a number of institutions. The IRB application for exempt projects is usually faster than standard IRB applications. For more questions on IRB or to see sample exempt applications, please email us.
Interview Script
| Tip |
|---|
The Interview Script is a document that the interviewer interview team can follow to conduct the interview. It contains an introduction, the questions to be asked of the interviewee and topics to focus on in observation.
...
| Panel |
|---|
The main focus of our interview today is to understand more about why and how you use scholarly resources in preparing and conducting your courses. Scholarly resources can mean anything you use for lectures, assignments, research or other course activities; not just materials you get from the library. We consider you the expert at your work so there are no wrong answers to any of our questions. While you answer questions or guide us through tasks, please focus on the details of how you usually actually do your work. It may help to think about the last time you performed the task and explain it to us as if we are going to need to perform the task soonjust as you did. Please feel free to be honest and critical even if the way your work actually gets done is not the way it should be done - again, we are testing the systems you use, not you, and everything you tell us is strictly confidential. Any questions before we begin? |
Interview Questions
Interview Phases
The interview questions can be broken down into the following sections:
- Warm-up: getting to know the instructor a little bit better and giving them some simple questions to get them feeling confident.
- General Questions: learning a little bit more about the goals, attitudes and activities of the instructor with more open-ended questions.
- Observation: focusing in on a specific activity and having the instructor think aloud, sharing details of why and how they perform tasks that comprise an activity.
- Follow-up: asking questions to clarify missing, ambiguous or confusing information from observation.
- Wrap-up: asking questions about the interviewing process, anything they would like to add and if it would be all right to contact them with follow-up questions or the opportunity to give feedback on prototype designs.
Format of a question
The format of the questions in our Interview Script are as follows:
[type] Question to be posed to instructor. Topics to cover, potential follow-up questions, focusing questions.
Example:
[Motivation] What do you enjoy most about your job as an instructor? Why? What activities do you always tackle first?
Types of questions:
- [Variables] - Verifying demographic and behavioral variables
- [Mental Model] - How does the instructor think about work, resources, services & systems?
- [Activities] - What kinds of activities does the instructor perform? Do they occur regularly or occasionally?
- [Motivation] - What are the instructor's goals? What does the instructor enjoy doing?
- [Opportunities] - What are the instructor's pain points? What do they avoid?
- [Interactions] - Who or what does the instructor consult or work with?
- [Process] - How does the instructor make decisions and perform tasks?
| Panel |
|---|
Warm-up Questions[Variables] How long have you been at <instructor's institution>?
General Questions[Variables] About how many faculty are in your department? [Variables] How much time do you commit to teaching versus other job duties? [Variables] Do you primarily teach small (up to around 30 students) or large (much more than 30 students) courses? [Variables] Do you primarily teach undergraduate or graduate courses? [Mental Model] How do you define scholarly resources? What kinds of things do you consider scholarly resources? [Mental Model] Are there other resources you use regularly for your courses? General QuestionsWhat kinds of things do you do[Activities] Could you please describe a typical day for you at work? Which activities occur regularly? Which activities occur only on occasion? How does course preparation and conducting a course fit in? [Motivation] What do you enjoy most about your job as an instructor? Why? What activities do you always tackle first? [Opportunities] What do you enjoy least about your job as an instructor? Why? What activities currently waste your time? Please think[Activities] Can you tell us more about the course you are currently most involved with (the same course as above). Can you give us an overview of how you prepared for this course?
? (If there is no definitive course: please tell us about a course you are currently teaching or, if you are not currently teaching, a course you most recently taught.) [Activities] Is this a course you regularly teach? [Variables] How many students are there? [Variables] Are there multiple sections? [Interactions, Process] Do you work with other instructors (including students) to prepare for the course before the start of the course? What do you do together? [Interactions, Process] Do you work with other instructors (including students) to conduct the course once it has begun? What do you do together? [Activities] In which course activities do you make use of scholarly resources? [Process] For each activity, what scholarly resources are used? [Process, Motivation] For each activity, how do you choose scholarly resources? How do the scholarly resources add value to the activity they are a part of, if at all? |
Observation
In this part of the interview, the interview team acts as apprentices observing as the instructor walks through an activity, interrupting with questions only occasionally. The goal is for the interview team to be able to re-create the work of the instructor. The activity of interest should be one that the instructor has said they perform regularly in preparing or conducting their course and scholarly resources are used in the activity. Try and select the activity that the instructor does most regularly.
For example, the instructor may regularly use journal articles as reading assignments when building a syllabus in preparation for a course. Given this example, the interview team may request the instructor give a running description of how they set up a syllabus:
| Panel |
|---|
We would like to observe as you give us a running description of how you set up the syllabus for your course. Please walk us through this activity just as you performed it last, thinking aloud and letting us know why you are doing what you are doing. Think of yourself as a master and us as your apprentices: we want to understand how you do your work so that we may re-create it. |
There are four kinds of information the interview team should be paying attention to while observing the instructor work (from Observing the User Experience, pp. 171-172):
...
Be sure to have the instructor focus on just one activity, think aloud and actually complete the activity. If time does not allow for the completion of the activity, be sure to ask how the instructor would finish the activity.
Areas to focus on during observation:
- Sequences - in what order is the instructor performing tasks? Are there interruptions? Are they doing more than one thing at a time? We want to be able to create rich workflow models (exactly how the user completes certain tasks) from this kind of data
- Tools - what is the instructor using to perform their work?
- Problems - what is frustrating or unexpected for the instructor?
- Mental models - how does the instructor think about their work? What expectations do they have?
- Interactions - do they consult different people, reference materials, processes? If so, what for?
Interrupting the instructor during observation:
The interview team should interrupt the instructor only in the following circumstances:
- The instructor is doing a lot of work without speaking about it – Ask the instructor questions about what they are doing and remind them to give a running description of what they are doing so that you, the interview team, may be able to re-create their work.
- The instructor is giving very general or broad details about an important task or action – Ask the instructor questions about what they are doing and remind them that you are looking for details of their work so that you can re-create it. Remind them that they can recount the details of the last time they went through this activity to make it more concrete.
- It is unclear what the instructor is talking about – the instructor may use jargon that the interview team is unfamiliar with. As an apprentice, it is important to know what you are dealing with and it is all right to interrupt the instructor to ask a clarifying question when you do not know what they are talking about.
Follow-up Questions
In this part of the interview, the interview team takes time to reflect on the observation:
- Are there gaps in the process the instructor has described to you? Now is the time to investigate and make sure all the steps involved in completing the activity are recorded. The interview team may end up delving back into observation mode as the instructor remembers something they forgot to mention and begins another description of tasks and actions.
- Are there other parts of the process that are unclear? Now is the time to clarify.
- Were there instances where the interviewer or note-taker had a question, but did not ask because it would have adversely interrupted the instructor? Now is the time to ask these questions.
Wrap-up Questions
| Panel |
|---|
Give the interviewee a big Thank You! What did you think of this interview? Anything you did not understand? Anything that made you anxious? Is there anything else regarding use of scholarly resources in preparing or conducting your courses that you would like to add? Would you be willing to be contacted with follow-up questions? Would you be willing to be contacted about providing feedback on prototypes we design? |
| Anchor | ||||
|---|---|---|---|---|
|
Note-taking
More info coming soon...
Post-interview Debrief
...
| Tip |
|---|
Guidelines:
- Try and capture every word the user says!
- Take notes directly on a laptop if the note-taker is a fast typist
- Taking notes by hand (can be useful in creating diagrams) or making an audio recording are also suitable options if the note-taker is more comfortable with these methods. We do require that interview notes shared with the rest of the group be typed up. Taking notes by hand or audio recording and then transcribing them may take a significant amount of time.
- Abbreviations are helpful, paraphrasing is OK
- Try and record especially poignant quotes directly and mark them as quotes (i.e. "The library is too overwhelming to approach cold.")
- Raw notes do not need to be perfect
- Clean up notes during debrief soon after the interview
- It's OK for the note-taker to ask to repeat something or ask for clarification (without interrupting the instructor too much)
- in between questions are a good time
- If the interviewer comes up with new questions, record them in the notes
| Anchor | ||||
|---|---|---|---|---|
|
Post-interview Debrief
- Arrange a debriefing session as soon as possible after the interview
- Discuss and record your initial general impressions from the interview in the Debriefing Notes section at the end of the Interview Script
- Save a second file (Save as...) as the "clean" version of the notes
- Go through the notes removing any personally identifiable information (names, non-work related places, etc.)
- Names/places should be substituted with anonymous titles:
- Gaurav Bhatnagar helped me ... --> [a faculty colleague] helped me ...
- Names/places should be substituted with anonymous titles:
- Go through the notes adding text to make the notes understandable by those outside of the interview
- Complete fragmented sentences or thoughts
- Expand abbreviations
After debriefing with your interview team:
- Upload your cleaned up, anonymized notes at the Interviewer Group Google Group.
- Email the Data Analysis Group with 1-hour times over the course of a week that your interview team will be available to have a phone conference to review the interview. Depending on the notes and the complexity of the interview, we may not need to meet for the full one hour.