Talking to users about learning activities

We often get feature requests and code submissions in JIRA, our bug tracking system. Issues submitted in this way are discrete, out of context, and highly specific to a particular users' need.

This exercise is different. Rather than collecting a list of feature requests from various people, we want to understand particular types of users and their goals. We will be contacting members of the UX, User, and T&L communities, as well as those who participated in the Multi-Institutional Survey Initiative to help us understand our users, however, anyone with an interest may participate. If you are interested, be sure to leave your name and contact information below so we can contact you about meetings and updates.

Who do we want to learn about?

At this point we hope to have at least 12-18 distinct user profiles from 5-6 schools.

• 6-8 Activity Creators (instructors, instructional designers)
• 3-5 Activity Takers (students)
• 3-5 Activity Evaluators (instructors, TAs)

Some of the people you talk to may fall into more than one of these categories. Beyond diversity in roles, we also want to insure we talk to people in different teaching situations and pedogogical approaches. We will be posting a matrix of the range of diversity we are hoping to cover shortly, for comment and discussion.

This diversity is important because we will be using your profiles to create personas -- distinct, archetypical users that can help to inform design decisions. You can learn more about how personas will be used in this page describing the Goal-Directed Design methodology.

How should we learn about these users?

We need people to interview people in various roles on your campus. These volunteers will need to

• recruit appropriate interviewees and schedule interviews
• conduct face to face interviews with instructors, students (as well as instructional designers, TAs)

The ethnographic interview we are doing is called a Contextual Inquirybecause it ideally takes place in the interviewee's work environment. Few people at Sakai campuses have people with the actual title of user researcher. We are simply looking for people who are observant, empathetic, and able to ask open-ended questions who are able to find appropriate contacts among end users; if you ever thought of being an anthropologist, counselor, or investigative reporter, this might be a good fit. We will be describing guidelines for conducting contextual inquiries (interviewing people in their work environment) during our kick off meeting.

What is the time commitment?

We would prefer that you are able to conduct at least one instructor interview, plus an interview with a user in another role (student, TA, or instructional designer). It should take a total of 6 to 7 hours if you conduct one interview, and an additional 3 to 4 hours for your second interview. The time breaks down as follows

• 1 hour for  Kickoff Meetingon September 29 (click link for details!), 1 hour for more instruction on recruitment and how to do a contextual inquiry.
• 3-4 hours per interviewee
  • 1 hr scheduling and 1 hr conducting an interview between October 7-October 23 September 30 and October 16
  • 2 hours writing a user profile based on your interview before October 28 23
• 2 hours sharing your profile with the team at a joint meeting on or around October 2723 or November 3

Institutional Review Board considerationsh4. Institutional Review Board considerations

Most campuses have an Institutional Review Board (IRB) that governs research with human subjects. Although this project involves research with people through interviews and observations, we believe this would be considered a quality improvement project (activities designed to continuously improve the quality or performance of a department or program, where the intention is not to share results beyond that department or program), which many IRBs would consider exempt.  Nonetheless, since every IRB is different, please contact your campus IRB as to whether your they would consider this endeavor  generalizable research with human subjects. They may want to know the following

• What is the project: We are conducting these interviews to learn more about how users create, manage, take, and evaluate learning activities, such as homework, surveys, quizzes and tests. Our findings will be used to improve the capabilities of the learning management system Sakai. These interviews will be used to create composite user descriptions that represent distinct user types. These composite users, called personas, will be used to drive design improvements in future versions of Sakai. The people who will review these persona descriptions are either those involved in the interview process, or members of the Sakai community that would potentially implement the designs in future versions of Sakai. Interviewers and institutions will be credited for participating in the interview process, but we will not indicate which schools provided the interviews that influenced a particular persona.

• Confidentiality: Interview notes will be shared on a site restricted to peers who are conducting similar interviews on other campuses and will not be published or presented individually. Subject names should not be submitted with the interview notes. Notes will include school, department, and type of class but no personally identifiable information. We will be asking interviewees to sign a consent form, which you are free to adapt as needed to your campus.
• Who will be interviewed: Instructors, students, and other roles involved in the creation, management, taking, and evaluation of learning activities. We will not be interviewing anyone under 18 years of age.

Most campuses have an Institutional Review Board (IRB) that governs research with human subjects. Although this project involves research with people through interviews and observations, we believe this would be considered a quality improvement project (activities designed to continuously improve the quality or performance of a department or program, where the intention is not to share results beyond that department or program), which many IRBs would consider exempt.  Nonetheless, since every IRB is different, please contact your campus IRB as to whether your they would consider this endeavor  generalizable research with human subjects. They may want to know the following

• What is the project: We are conducting these interviews to learn more about how users create, manage, take, and evaluate learning activities, such as homework, surveys, quizzes and tests. Our findings will be used to improve the capabilities of the learning management system Sakai. These interviews will be used to create composite user descriptions that represent distinct user types. These composite users, called personas, will be used to drive design improvements in future versions of Sakai. The people who will review these persona descriptions are either those involved in the interview process, or members of the Sakai community that would potentially implement the designs in future versions of Sakai. Interviewers and institutions will be credited for participating in the interview process, but we will not indicate which schools provided the interviews that influenced a particular persona.

• Confidentiality: Interview notes will be shared on a site restricted to peers who are conducting similar interviews on other campuses and will not be published or presented individually. Subject names should not be submitted with the interview notes. Notes will include school, department, and type of class but no personally identifiable information. We will be asking interviewees to sign a consent form, which you are free to adapt as needed to your campus.
• Who will be interviewed: Instructors, students, and other roles involved in the creation, management, taking, and evaluation of learning activities. We will not be interviewing anyone under 18 years of age.

Can I just tell you about our users?

It's not our first preference. There may be certain things we want to learn about which you've never explicitly talked about with your users. We don't want you to guess on their behalf, so we hope you can take the time to interview them. However, if you absolutely don't have time to do an original interview, we're interested, as long as you can tell distinct, detailed stories about particular users, rather than a blended average. This is particularly true if we've already recruited our minimum number of interviewees.

Can I send a questionnaire rather than interview them?

Again, it's not our first preference. With a questionnaire, you will not be able to observe the frustration or excitement on someone's face as they describe something, see the series of notes tacked to their bulletin board to remind them of important dates or the word documents they mail back and forth, or be able to follow up with them on a random comment they make. While questionnaire are good for getting wide coverage and possibly minimizing the note taking on your end, we would rather have one or two detailed profiles than five or six questionnaire responses.

I'm interested in participating. Where do I sign up?

We will be posting interview guidelines in mid-September, but wish to have a live meeting with those who are interested to kick off this project.